Natanox

Mmh, seems like that project is somewhat in license hell. Awesome that person does it anyway though, better downloading the releases before Nvidia gets pissy.

Oh god, it’s Matrix Element. Yeah, of course it’s absolute shit.

My question would be if it uses stuff like CommonCrawl as training material. Given their size I assume they used anything incl. unethical training data, but at least admit it?

The only models I ever found that even tried to only resort to ethically obtained data, being FOSS etc. were the tiny ones from PleIAs. And as expected they’re completely useless.

So far I concluded that an “ideologically sound” LLM is impossible due to lack of training data. Unless your ideology allows to steal stuff.

It’s even more funny if you know the backstory of bun. It was basically the perfect example of how to not to do things before they burned a whole forrest to have Claude slop it into another language, based on a codebase that was measurably bad. Now nobody actually knows the whole codebase anymore.

Given the phenomenal garbage that are the other big vibecoded projects (like the “compiler that can compile Linux” - it can’t) using bun now is just like lottery.