Turns out Generative AI was a scam

Article URL: https://garymarcus.substack.com/p/turns-out-generative-ai-was-a-scam Comments URL: https://news.ycombinator.com/item?id=47161111 Points: 6 # Comments: 0

Once the public has decided to accept something as an interesting fact, it becomes almost impossible to get the acceptance rescinded. The persistent interestingness and symbolic usefulness overrides any lack of factuality. Remember how in November the White House Crypto and AI advisor was telling us that Generative AI was contributing half of US GDP growth? I would love to see a companion graph about how much the banks are now tied to all this.

The piece by Ovide is absolutely brutal, somewhere between tragic and comic, filled with interviews showing how some hard to swallow numbers that fit a Silicon Valley narrative rapidly became gospel in Washington – with far too little scrutiny. Not since Geoff Pullum’s Great Eskimo Vocabulary Hoax have I read such a deconstruction of people hearing what they wanted to hear. None of what Ovide had to say about the overestimation of Generative AI should actually come as a surprise.

Generative AI has been inherently unreliable from the start; none of the problems that I warned about over the last half decade has been properly solved. Large language models still hallucinate, and they still make boneheaded errors; they still lack a proper concept of reality. They often produce workslop. A recent survey called The Remote Labor Index found that they could only do 2.5% of human tasks, and that is a massive overestimate, since literally everything that requires physical labor was excluded.

Generative AI has its uses, but it hardly surprising that most business have struggled to find return on their investment, given the massive reliability issues that continue to plaguage it. And so the economic impact may well be a lot smaller than many people were led to believe. I don’t know that Generative AI was literally a scam, but the people selling it have tried to sell it as if it were tantamount to artificial general intelligence, when it’s not.

And there is just no way for the product they actually know how build — as opposed to the one they fantasize about — to live up to those expectations. When all is said and done, my best guess is that generative AI will have done significantly more harm to society than good. Although there are some practical use cases, such as coding, it is an inherently unreliable technology. It is ripping apart our educational system and our information ecosphere, and flooding the zone with nonconsensual deepfake porn.

It is threatening the environment with data centers built on too much speculation. It is leading some people into serious mental health issues. And it may well lay waste to our economy, once banks and investors who bought the hype start to fall. Will you PLEASE stop saying "coding is a practical use case"? This is the third appeal I've made on this subject. (Do you read your comments?) If you want bug ridden code with security issues which is not extensible and which no-one understands, then sure, it's a practical use case.

Just like if you want nonsensical articles with invented facts, then article writing is a practical use case. But as I've pointed out already no reputable editorial is now using LLMs to write their articles. Why is that? Because it obviously doesn't work. Let's face it the only reason you're saying "coding is a practical use case" is because you yourself don't code, and don't understand it. I can't see another reason why would assume the problems experienced in other domains somehow don't apply to coding.

Newsflash: they do. And software engineering definitely doesn't need the slop any more than anyone else. So I hope this is my final appeal: please stop perpetuating this myth. If you want more information on the problems of using LLMs to code, then I can talk in great length about it – feel free to reach out. Thanks UGH that was with model n, model n+1 that has 3% higher score on a benchmark that has test set completely contaminated will solve everything.

Now just hand over 300B you damn luddites.

Summary

This report covers the latest developments in artificial intelligence. The information presented highlights key changes and updates that are relevant to those following this topic.

---