- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.
Hope you enjoy this reading, all feedback is welcome.
Neural nets are dumber than you thought for sure. What I mean is the coin flip results are known after the fact.
I’m not convinced of that you do. Nor is knowing that flip helpful. Because it’s non deterministic. An internal coin flip is not an input either.
What I mean is for post hoc analysis since you know the coin flips and the inputs you can step through the layers and nodes calculation deterministically.
I mean sure, but to what end?
You can see where it went wrong. Not why, how
You can try to reason about it which is an arguable improvement from the former engineer you can’t get ahold of anymore. So you can could see that it triggered a node associated with c++ when it generated this string of tokens or compare it other generations to map it to a type of call.
Where it doesn’t compare to a human engineer at the moment is quality or novelty.
But you can’t?
It didn’t reason its way down a branch, it made coinflips all along the way. So its not actually thinking, just weighted coin flips. There’s no ‘why’ it chose a branch. Just the result of a coin flip.
You can’t ask a coin why it chose heads. You can unroll the physics of it all day long til the cow comes home. But it still doesn’t help you on the next one.
It’s not really coin flips it and random adjustments of weighted values that decides which nodes are activated which, plus random adjustments, influences the next node activation until out
Statistical outputs are not as interesting as you think?
Like sure, but its still just a probabilistic outcome. Not one reasoned at any step.