spoiler
When I say LLMs I always mean opensource, openweight models not OpenAI/Anthropic/X-AI crap, these companies are trying their darndest to spread misinformation and lobby to quell and prevent the development of these opensource models, they should be punished by creating an alternate ecosystem untouched by the opinions and whims of these feudalistic tech lords
The exam/criteria should be broad and rigorous, The license issued and the exam should be private(tied to a handle not PI name), unique and publicly verifiable.
Right now the onus is on individual projects to take a stance for/against LLM use for any contributions in their codebase.
With all the opinions floating around regarding LLM use for programming(most of which i feel is either misinformation, or technophobia), I think this would at least stem some of the legitimate concerns regarding competency and responsibility.
drown out the slop. If a slop kiddie makes a 600+ diff PR, you can safely ignore it, but if a license holder makes a PR you consider it.
how does this compare with something like the vouch system, possibly with some sort of federated mechanism, so you could choose which list of users to trust
It might discourage new contributors and create a sort of chicken and egg situation, also a vouch system has the inherent risks of being camped by bots
I think having to get a cert is even worse.
and some will get camped by bots, but that’s fine since different projects have different tolerances for contribution quality. I’m thinking like the bluesky moderation system, where each repo can subscribe to a particular pool of vouches in addition to their own. you may have lists for no ai, lists for responsible ai/abides by repo rules, or good intentioned but pure ai and minimal human contribution.
How do you identify a “slop kiddie”? It’s not like LLM users identify themselves as such.
You’ll have to judge by their code, at which point you’re already not ignoring them at all.
in my mind a slop kiddie is no different than a script kiddie, both use the tools they have found with no understanding of how it works, both rarely contribute anything meaningfully new.
as such there are tells, of course I am not suggesting we ignore every anonymous PR, but the ones that broadly fit into certain patterns