Of AI and D&D
I have always enjoyed high fantasy stories and role playing video games (especially the Zelda series). But tabletop role playing games, namely Dungeons & Dragons, were nowhere in my life. Then several years ago my brother and a couple other guys invited me to learn it with them and play. I was hooked immediately. It would not be an exaggeration to say during that first year that I created well over a hundred characters, even though I only used one of them… the character building process is just that enjoyable for me. But with a family, a job or two, other interests, and all the responsibilities of modern life, it is hard to get together regularly and play. There’s a million memes about that struggle.
At the start of this year, I found a solo RPG app – a text-based game in the vein of D&D. I downloaded it, I tried it… and it was really fun! It was not sitting around the table with my adventuring party, it had no social component – it was just me talking to a large language model and then it describing an adventure to me. It was not a replacement, but it did help scratch that itch on the weeks that I couldn’t get together with my friends and play. I could play about one hour a month using the free tier. So, I bought a 6-month subscription. I do surveys through the Google rewards program and it gives me Play Store credits (referral code if you would like to sign up: 499L6H8K), which I then save up until there’s something I want to get (they don’t expire for a whole year once they’re issued), so it cost me less than $20. But the more I played it, the more I began to notice some problems. I can group these problems into four buckets:
- Narrative Drift
- Inconclusivity / Impreciseness
- Data Connection
- Subscription fees
But I was willing to put up with those things, until they released an update that radically changed the way Non Playable Character (NPC) interactions worked, and dramatically increased the amount of messages required to play the game. The subscription tier I got was the lowest one; even their highest one ($50/month at the time, more now) was not unlimited.
I would say “Artificial Intelligence” is a big deal these days, but people rarely use the full term, they prefer the acronym “AI”. We know what it stands for, but by not saying “intelligence” we (collectively) find it easier to give it more credit than it’s due. Usually when people are talking about AI what they’re really talking about is a generative text, image, or video platform that relies on a trained model. For text generation it is a Large Language Model (LLM for short). These result in products like ChatGPT. These systems are very exciting… advanced technology that is really the next evolution of all the automation and search indexing and other technological endeavors that we’ve been making since the first electronic computer was invented. While being artificial, while being very impressive, they aren’t really intelligent. But they do have some really cool applications. And this was a really cool application of it!
Narrative Drift
A problem inherent to chat sessions with a large language model is context. Every chat session has what we call a context window. This context window is everything that that session “remembers”. The larger the context window, the slower it moves. And, the more opportunities there are for contradictory pieces of information to be included in the “thought” process. So sessions will compact the context. Basically, the system goes through and attempts to determine the important bits that it needs to carry forward into the next part of the session, summarizes some of it, and dumps all the details. So, the further away from any stated piece of information, question that was asked, or decision that was made, the less likely it is to know about it. In the game, this led to things like an NPC switching races in between scenes – they might start out as a halfling (hobbit) and then end up being an elf – or characters just disappearing from the story altogether. As a player, that is incredibly frustrating. Also, it would cause it to conflate goals. I might be exploring a cave and come across a cult that I destroy, and then later be in a town questioning some thug about some crime they’re doing, and all the sudden the game connects the two things and the cult is back in the picture out of nowhere, and the story beat I was on has totally shifted – and not in a good “it was all connected from the start” kind of way, more like an “I guess we’ll make Prince Hans the bad guy instead” shoehorning.
Inconclusivity / Impreciseness
In a tabletop RPG the narrator is called a Game Master (GM) or Dungeon Master (DM). Dungeon Master is actually legally specific to just Dungeons & Dragons, but is a popular enough term that it gets used casually all over the place – like calling a tissue a Kleenex. And this GM had the habit of being vague. Because large language models are trained on existing literature to calculate the probabilities of word groupings, words, expressions, and themes that recur a lot in writings get preferential treatment. So rather than inventing unique, detailed narratives it was rehashing the same plot points over and over, and always being dramatic (purple). This is an inherent problem in language models and one that the game can only do so much about.
Part of AI not actually being intelligent, just giving the appearance of it, is that language models are not good at math. You may recall when ChatGPT was first getting popular and people would ask it simple arithmetic questions and it would get the answer wrong. People would laugh and say “Oh, what a dumb AI, it doesn’t even know what 6 + 7 is!”. Yeah, it doesn’t “know” anything and it is not designed to do equations, it was created to guess what the next word is supposed to be based on context clues (just a rough explanation), so when it sees ”6” and “7” in a text string, it feels pretty good about throwing “8” out there next. The chat platforms have gotten better at handling this kind of problem by mixing models and adding some layers to the process…. But there are still an inherent issue! So, when I would have 200 gold pieces, and then buy something for 10 gold pieces, and then I was left with 170 gold pieces, I would be ticked off.
Data Connection
I have an unlimited data plan. I have Wi-Fi at my house that uses the fiber Internet connection that I pay for every month. I’m not really worried about data. But when I’m inside of a waiting room with bad reception, or on a road trip in the middle of nowhere, or on a plane that doesn’t have an internet connection, I enjoy having an offline app. Often I’ll read (if it’s not too bumpy, as I age I have found reading in the car to be more problematic), but if I’m traveling with my kids and need to be able to put something down and then pay attention to that and then able to go back to my entertainment, I don’t really want to be reading a book – I either need quiet so that I can retain the information in it or time to really get into the story and enjoy it (depending on the type of reading I’m doing). If I’m with someone, talking is always nice, but there’s not always conversation to be had, so having an offline app is nice; sat on a plane between two strangers for 3 and 1/2 hours could be a good time to get some mental adventuring in! Alack and alas, that was impossible with all the apps of this kind on the market.
Subscription Fees
Do you remember when we bought things? Nowadays everything wants you to create a profile and get a subscription. I got a new phone a couple of months ago, and I, on purpose, did not transfer all my old apps and data to my new phone. I had five different parking apps on my phone! Parking in Galveston, parking in Houston, parking at the Johnson Space Center, parking in San Antonio, and one app that had gone out of business but I never uninstalled it after I used it one time and I don’t even remember where it was for. We used to just put coins in a meter or fold up some bills and put them in a slot, or even swipe a credit card on site. Not anymore! Now I have to give a company I have never heard of my email address, and then uncheck a box so I don’t get a weekly newsletter about parking, and set up multi-factor authentication so somebody doesn’t hack into my parking account… Yeah, yeah, I hear it – I’m an angry old man (I’m not even 40). There are DLCs for video games and expansion sets for board games; I’m good with both of these, they give me new content and new abilities and new facets of something that I enjoy, when they’re completely optional and bring additional value – so them carrying an additional cost makes sense to me. But I am tired of paying the monthly fee for something I might use for a couple of days in a month (or every other month), just in case, so that it’s there when I need/want it. Something mission critical, something that requires active monitoring, like a home security system, I can understand that. Something that is active all the time and you actually use even when you’re not actively using it, like cloud storage space, I understand – I put files there, if I want the files to stay there I need to keep paying for it.
These kinds of apps charge you based on usage, whether it’s time, tokens, or messages (which is the most popular one in my experience) because they are having to pay for usage of something like the OpenAI API (programmatic access to the functions of ChatGPT without going through the interface of ChatGPT). The problem got exacerbated when the main app that I had played changed their dialogue system, and messages got consumed even faster. What had been 8 to 12 hours of play per month (aid subscription), went down to 2to 3. And the narrative quality suffered right along with it.
So I had an idea: I’ll make my own.
