Training an AI on the Subtext of a Story

What it takes to make a difference in the quality of storytelling generated by artificial intelligence (and humans)

Over the past couple of months, I’ve found myself gravitating more and more towards Bluesky, a new social network built around the network’s protocols, rather than the company who built the network. With the demise of X/Twitter and any semblance of its usefulness to my work, I find the quality of conversation on Bluesky more suited to my demeanor.

One recent conversation kicked off around the idea of being able to train an AI on the subtext of a story, rather than the story itself. This ability is the key to Subtxt’s advantage over every other creative writing AI system out there. Training on the text of a story is guaranteed to fail every time--as its the message behind the words that we strive for as audience members (and as authors), not the vectorized embeddings of the words themselves.

I invite you to follow me on Bluesky (and, of course, follow Narrative First on Bluesky), as I’m pretty confident that I’ll be having many more conversations like the one below in the coming years.

Finding What Lies Beneath

You can train an AI on the underlying meaning of a work (subtext), rather than the work itself. Once you separate storytelling from narrative intent, you can focus on the latter and help authors better understand what it is they’re trying to say.

The way an AI works is it takes a word and attaches it to a number then it calculates the probability of what numbers are most likely to go next to each other. That's all it's doing. It's telling you what words are statistically likely to be next to each other, then adjusting the stats

Remember the other day when ChatGP "went mad"? It was because the number generator was slightly off so it was putting the wrong numbers in the sequence and thus generating nonsense words. The current model of AI are number generators, that's all. There's no subtext in AI writing.

What there is is the chance to analyze broad strokes patterns, which might be interesting in many fields but isn't useful to writers. Just because your words match a pattern doesn't mean you're communicating well. Writing to match the pattern is a good way to be boring.

I wasn't referring to pattern matching. The order of events in a story, regardless of how they're expressed, carries meaning. A slap followed by a scream means something different than a scream followed by a slap. You can train an AI on that underlying meaning, and that is helpful to writers.

But you can't though, at least not with how AI currently works. What you can do is feed a lot of text into it and find out which one people write more often. AI can't distinguish meaning. It can only generate text based on probability based on what texts you used to train it.

If you have a narrative framework specifically built around the meaning of a text rather than the text itself, then yes you can have an AI that distinguishes that meaning. I've already built it and its currently identifying the subtext of thousands of different stories.

Yeah I confess to confusion. The meaning of a text is communicated by the text and you can't really separate it. In some regards the "meaning" of text is always going to be up to reader interpretation to begin with. So how any AI, even AGI, could codify it is beyond me.

It kind of reminds me of how my a pair of english teachers at my high school would teach texts. Like the entirety of meaning was stuff like "the ducks Holden Caulfield feeds represent childhood" and that boiled down to them wanting an excuse to fail kids they didn't like personally.

You can always extricate storytelling from storyform (the underlying meaning, or argument of a story). _West Side Story/Romeo & Juliet _are best example. Different words, same story. Braveheart/Gladiator are another: headstrong warrior convinces politician to take action for country's future.

That's because genre is also a kind of category imposed by readers on the text from outside. It comes from outside experience and can influence how a writer works but you're not going to extract it from the text. Nothing fits perfectly into a genre because ideas about genre change but text stays

Genre isn't the same thing as meaning but they are both products of reader interpretation. No idea how you'd train an AI to recognize them and even if you could it wouldn't be helpful because again those ideas tend to change a lot over time

If you see Genre more as an indicator of various areas of conflict/inequity rather than drop-downs in Netflix (Comedy, Action/Adventure, etc.) you can start to build a mental model of narrative exploration through each (e.g., Slasher Horror on physical conflict, Psch Horror on psych conflict)

So you know how AI works? It records things and then reproduces them based on probability. You can modify those probabilities but the patterns AI picks out aren't the same thing as meaning. AI generates text by using it's training set to predict what set of words is likely to follow the previous set

When you claim to train an AI on meaning I don't know what you mean because that's not actually possible. You can train it to produce some words more than other words but that's pretty much the extent of it. AI can't think for itself, it can only reproduce what its shown based on statistics.

Yes, I know how AI works (intimately). I’m not talking about training on the meaning of text embeddings, but rather the underlying message the author communicates with their story (their argument/purpose/intent). Training on former, I agree, will always get you garbage.

How? Like are you deciding what an author means writing it out and then feeding it into the AI? What do you do about disparate interpretations of texts? If EL James screams "No rape in 50 Shades read the book!" at rape survivors do you take her word for it?

No, that would be too subjective. If you model narrative to include an objective POV to measure the subjectivity, you base the meaning on the story’s exploration of resolving inequity (conflict). With that context an AI can “read” a text and suss out the underlying argument (i.e., message).

There's no such thing as an objective POV to measure subjectivity. Most writing doesn't have an "argument". It has elements that people can analyze and take ideas from but different people are going to see them totally differently. Your AI is reading a thing that doesn't exist

I think you're doing the grammar check thing I tried to point out to the previous guy. You're trying to define an absolute meaning (arbitrarily honestly) and telling people who use your app that it's the only truth because you used a scientific model (that you can't explain)

By definition there can be no subjectivity without objectivity (two sides of the same coin and all that). And yes, my POV presumes author’s intent to argue the appropriateness of various paths towards resolving conflict. Most stories are built this way whether author knows it or not.