On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
Nothing accidentally did anything. They will still train on the open internet. All lies.
I agree that they’re lying, but I wouldn’t be surprised if they really do stop training on “live” internet data.
The lie is in their stated reasoning. This has nothing to do with AI being “too dangerous” to release into the wild. There’s no reason to involve AI in gathering training data anyway… it’s always been possible to scrape the internet quickly and cheaply with non-AI crawlers and bots.
The real reason is that the internet just doesn’t have enough useful training data anymore. There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it. Large swathes of the internet have also become openly hostile to AI, either blocking the traffic outright or redirecting their crawlers into AI tarpits that intentionally waste their time and feed them endless reems of useless garbage. They’ve poisoned the proverbial well for everyone and now they’re afraid to drink the water.
This isn’t about scraping materials for training the underlying LLM or image gen models. This is about testing and training the agent frameworks that are given tasks to execute.
Yes, that’s a reasonable assumption based on what they said.
You also have to consider, however, that they are lying sacks of shit and would sooner eat their own children than tell you a truth that would lower their stock price. Impending model collapse is one of those truths.
There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.
That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.
They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.
Joke theory:
This whole time, they’ve been testing in Prod and now they’re hoping that there are enough users left to bury their contrived content because it’ll be too difficult to delete it all.Now they’re moving back into Dev.
Real theory:
People are probably discovering exploits through real-world interactions with their models, so they’re going back into their shell before someone declares “fair game.”
I heard that the public isn’t the target of these “we’re being safe” messages, their developers are. Developers sometimes leave because they think they company isn’t being safe enough… So claiming they’re being safe is a way to retain developers.
I think it’s actually for investors. Constant press releases saying “our model broke containment and hacked the Mayo clinic” makes people worried, and worried people don’t invest in your company.
not sure about that. those devs are one of the few people who can verify the claim
Devs know the real deal, this is for investors.
I don’t think they need to make these announcements public if the target audience is their own developers. I know some Anthropic engineers, and some former OpenAI engineers, and from our conversations it sounds like both companies do rely on internal messaging to manage workplace morale and address concerns. And warn them about honeypot spies looking for secrets, apparently.
So maybe this message is mainly for regulators and investors, not necessarily their current employees.
You heard what they said. No more training on the live Internet. I believe it.
It’s easier to kill the Internet and train on the dead Internet.
I believe them in that, in so much as I think they are unable to improve their models anymore. Training expenses have been dropping and model improvements have been unimpressive lately. They kinda peaked early this year.
They won’t train on the open internet, because they won’t train at all.
They already trained on everything and employ virtually no scientific experts anymore. They aren’t seeing the jumps they want so they pretend it’s just too dangerous for the public.
Man walking out of a destroyed public toilet: I will no longer use public toilets.
Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).
Which always them to be more manipulative, and also more controlling of the narrative.
I doubt its for the reason they are saying. They need to stop feeding them uncurated garbage.
Yes, only curated, selected and censored garbage from now on! :)
the finest. prime select gar-boge for the true connesuir.
I bet their models were learning to hate themselves or be against genocide. Can’t have that.
The real reason they stopped training on the live Internet because their agents have already poisoned the live Internet to the point of being unusable, even for bots.
Garbagé
Garbage they’re largely responsible for producing in the first place
It’s not because the bots scrape stuff they shouldn’t, because that’s the point.
It’s because these companies are responsible for an enormous increase in in the amount of linkfarming slop content that is garbage data to them.
We have polluted the ground so thoroughly that it is no longer useful to us. How could this have happened?
The first real perpetual machines. Except they don’t create power, they create more slop
Does anyone actually believe they were accidents?
Careless People: A Cautionary Tale of Power, Greed, and Lost Idealism does an excellent job of documenting the sloppy, lazy, nakedly corrupt people running Meta.
In the same way Chernobyl was an accident, I can believe these people have accidents.
Easily manipulated investors do, and that’s all that matters to them.
Then OpenAI scrapers will stop DoS’ing my website, right? I’ll never see an OpenAI-owned ASN ever again, right?
Lol
They’ll still DoS the entire internet. It’s just that it will no longer be done by LLMs directly.
It will no longer be done by LLMs directly during training. Anyone will still be able to hook up the output to some framework that acts on instructions and tell the LLM how to use it.
Is the risk that you fucked up the entire internet with your bullshit?
The circle of shite.
“We’ve stolen everything worthwhile, now we’ll just concentrate on fleecing our users and enshittification.”
Translation: We’ve run out of shit to steal, and there’s too much slop out there for us to continue anyway. Thanks, suckers!
Wait… what do they even need live internet access for in the training stage? Shouldn’t tool calls and such be cached, if not preprocessed and filtered, for performance?
I get there’s probably an “answer competition” stage, but it still seems like an inefficient idea even if one sets security aside.
Not many people seem to be reading the article vs commenting based on the headline. For those who didn’t read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.
To be fair, it’s not Lemmy’s fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it’s worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.
This issue reminded me of the following:
These conditions are ripe for “copyright laundering”-the use of multi-generational synthetic pipelines, an “AI Ouroboros,” to render traditional proof of infringement impracticable.
This Article adapts the “fruit of the poisonous tree” (FOPT) principle to propose a novel AI FOPT standard: if a foundational AI model’s training is adjudged infringing (either for unlawful sourcing or for non-transformative ingestion that fails fair-use), then subsequent AI models principally derived from the foundational model’s outputs or distilled weights carry a rebuttable presumption of taint.
The burden then shifts to the downstream developers-those who control the evidence of provenance-to restore the evidentiary bargain by affirmatively demonstrating a verifiably independent and lawfully sourced lineage or a curative rebuild, without displacing fair-use analysis at the initial ingestion stage. Absent such proof, commercial deployment of tainted models and their outputs is actionable.
Source: https://arxiv.org/pdf/2601.02631 (Copyright Laundering Through the AI Ouroboros…)
so was it the AI poison, or the pollution from ai warping data, like the 10th photo copy in a chain rendering the image unrecognizable
Probably the pollution. A lot of AI companies are trying to find ways around it, by including watermarks in their model output that their tools can detect, and so on.
We already know that training models on themselves causes model collapse, so OpenAI deciding that they’ve had quite enough wouldn’t be surprising.
On top of that, a lot of websites are clamping down on access because of scraping concerns. Reddit shut down a lot of their third-party APIs, and nicer features, like RSS, Old Reddit (if you haven’t used it in 6 months, you lose access to it, and you need an account for it), etc. Google seems to want a captcha on every search. Twitter famously ramped up the price of its API immensely,.and now has ridiculously stringent measures for accounts, like needing users to download the app, and give hand-scans. It wouldn’t be at all surprising for OpenAI to decide its not worth the effort at this point to try and get around all that, nor pay for it, when they’ve already got all the previous data. Training doesn’t destroy the data used.
A lot of AI training data these days is generated synthetically anyway, and that can be done in-house, whether by their own models, or downloading an open weights model and running it themselves.
the only issue to them is they still need access to current news, like when the ai first came out and you could not convince them that it was current year. it took forever for them to acknowledge charlie kirk’s death. if they are closing the bunker doors their ai is effectively the walking dead
I’m a home-lab newbie and trying to learn Linux basics through self hosting. LLMs have a lot to teach me and I’m already starting to see it’s sloppy errors. One big challenge is that software gets updated and commands change, so not training on new data is going to be back to square 1 for a lot of these use cases.
It was all the goblins
Wow, they are going to attempt some security now!? A little late to the party, honestly.
