• Pup Biru@aussie.zone
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 day ago

    assuming you need to be fast that’s true, but if you don’t care how much time you take then you can spread the processing out… this is exactly how MoE models are much easier to run locally, but you could go much further

    • AVincentInSpace@pawb.social
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 day ago

      I’m sure the answer that comes out of your throttled CPU only language model in a month is still going to be relevant.