The developer behind Teach My Little Sister How to Drive, a driving instruction sim where you instruct a generative-AI-powered, dynamic NPC, has been spending $1,000 dollars a day to keep the demo working.
The developer behind Teach My Little Sister How to Drive, a driving instruction sim where you instruct a generative-AI-powered, dynamic NPC, has been spending $1,000 dollars a day to keep the demo working.
If they’re actually hitting a cloud LLM, there’s no good reason not to wrap that and memoize the results… it would probably be hard to generalize the inputs (it could just be the whole game state for all we know 🤷♂️) but youd build up that database of all the most common input/outputs in real-time
Exactly, that’s just smart programming (maybe that’s one of the issues… but I can’t say that without evidence).
I saw someone years ago develop that concept for a text to speech idea for NPC responses, and for the same reasons. Why repay for something when you can just capture the sound file for each new phrase? The cost is in the creation and addition of things, not in the actual running of every person’s game (or app in this case).
That would require you to actually understand what your code is doing…