The Cost of Software
jeferson.me
Software is costly and mechanical parrots won't make it cheaper. Do you want to know why?

As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.

Hope you enjoy this reading, all feedback is welcome.

  • fruitycoder@sh.itjust.worksEnglish
    12·
    2 days ago

    First agreed. Honestly I haven’t an LLM do anything close to an engineer. You can get lucky for a code snippet or two and maybe a few ADRs but even those are fraught with the chance of slop clean up work. I am not arguing that.

    The node is only probalistic because of the random number added. After the fact the number is known.

    • Shin@piefed.socialOPEnglish
      1·
      1 day ago

      So you are suggesting that we should have the metadata for every decision and direction. This is a N^N amount of data. Keeping this data is wastefull (even more than the usage of LLM right now). Not saying that is is useless, but for sure this won’t help to mitigate the cognition since this data don’t carry meaning for us humans.

      • fruitycoder@sh.itjust.worksEnglish
        11·
        15 hours ago

        No that should be discoverable with the models weights, input and random numbers added to the weight at the time.

        Say for example you find that a collection of outputs behave oddly or in an undesired way, you could use this to find what simularties they share with each other but delta with other and naively prune the nodes or simply decrease their weights. Those you could also try to corralate that to certain input tokens to engineer better prompts or try to trace it back to initial training data.

        It could also be a failed tool call adding garbage data in, or malicious. A trace could catch that as well.

        • Shin@piefed.socialOPEnglish
          1·
          7 hours ago

          No, the random part happens in the query.
          There is also random in the training, but the query also generate more random numbers.
          Otherwise this would be a deterministic procedure, and it’s not.

          • fruitycoder@sh.itjust.worksEnglish
            1·
            3 hours ago

            We are saying the samethings but you are adding no to it.

            Right, during inference random numbers are generated, that plus the numbers from input are added to the weight values and the matrix multiplication happens. If you used the same random numbers and inputs it is deterministic. For regular use you don’t do that because you want a stochastic output, if you wanting to do forensics and trace what led to an output you would benifit from that determinism.