- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.
Hope you enjoy this reading, all feedback is welcome.
So you are suggesting that we should have the metadata for every decision and direction. This is a
N^Namount of data. Keeping this data is wastefull (even more than the usage of LLM right now). Not saying that is is useless, but for sure this won’t help to mitigate the cognition since this data don’t carry meaning for us humans.No that should be discoverable with the models weights, input and random numbers added to the weight at the time.
Say for example you find that a collection of outputs behave oddly or in an undesired way, you could use this to find what simularties they share with each other but delta with other and naively prune the nodes or simply decrease their weights. Those you could also try to corralate that to certain input tokens to engineer better prompts or try to trace it back to initial training data.
It could also be a failed tool call adding garbage data in, or malicious. A trace could catch that as well.
No, the random part happens in the query.
There is also random in the training, but the query also generate more random numbers.
Otherwise this would be a deterministic procedure, and it’s not.