How Subtxt Uses Evaluations to Build Smarter AI for Better Stories

Smarter AI, sharper stories, constant evolution

At Narrative First, we believe storytelling is as much an art as it is a science. That’s why we’re constantly refining the tools we offer to help you bring your stories to life. A big part of this process is evaluating and improving the AI models that power features like BrainstormingAI. These evaluations don’t just measure success—they guide us toward smarter, more insightful storytelling tools.

Recently, we compared two of our AI models—the Standard model and the Advanced model—to see how they measure up in delivering narrative insights. These evaluations don’t just highlight what’s working; they reveal where we can dig deeper to improve. Here’s what we found—and how it’s shaping the future of Subtxt.

Testing What Matters Most

Every story has a structure, a progression, and a reason to exist. To ensure our AI understands these elements, we evaluate it across four key areas:

1. Method as a Source of Conflict: Can the AI recognize how a character’s approach to solving problems creates tension in the story?
2. Understanding the Definition of the Method: Does the AI grasp what the Method really means and how it operates in context?
3. Avoiding Conflict Pitfalls: Can it avoid common storytelling traps, like substituting vague conflicts ("struggling to," "threatening to") for meaningful tension?
4. Development of Story Context: Does it help build and deepen the narrative as the story progresses?

These criteria ensure that Subtxt’s AI isn’t just generating ideas but genuinely helping you create narratives that resonate.

The Results: Advanced Model Outperforms, But There’s More to Learn

Our evaluations revealed that the Advanced model consistently scored higher than the Standard model across most criteria. For example:

• In Avoiding Conflict, the Advanced model scored 92%, outperforming the Standard model’s 85%. This means it’s better at identifying direct and meaningful sources of tension.
• In Understanding the Definition of the Method, the Advanced model nearly aced it with 99%, compared to the Standard model’s 97%.

But even with these wins, there’s always room for growth. When it came to the Development of Story Context, the Advanced model scored slightly lower (81%) than the Standard model (82%). This told us something important: while the Advanced model excels at identifying and defining conflict, there’s still work to be done in helping it guide users through the nuanced progression of a story.

Subtxt Is Learning From Subtxt

These evaluations aren’t just numbers on a chart. They’re insights into how Subtxt can improve itself. When we see areas like Development of Story Context fall short, it sparks a deeper dive into why. We analyze the feedback loop between the AI and your stories, looking for patterns and opportunities to teach the AI how to better understand the subtleties of narrative progression.

And this is where Subtxt shines. By using Subtxt to evaluate itself, we’re creating a system that’s constantly learning and improving. It’s like having a story development and narrative design expert who’s not only getting better at understanding your needs but is also teaching itself how to be the best storyteller possible.

What This Means for You

For you, this all translates into smarter, more reliable tools that feel like a true creative partner. Whether you’re using BrainstormingAI to refine your character arcs or to pinpoint thematic nuances, these continuous improvements mean you’ll always have the best possible insights at your fingertips.

We’re not just building tools; we’re building a smarter, more intuitive system that evolves with every story it touches. Subtxt isn’t just learning to think like a storyteller—it’s learning to think like you.