Problems with AI, 2025 and Beyond

[Tintin]

This post deals with AI's inability to determine what's fake from real, in what could best be described as really very ironic indeed

It couldn't tell a joke either, I suspect.
________________________________________

Source: https://x.com/HedgieMarkets/status/2042430442448548273

🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks.
The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people.

ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher.

My Take
The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing.

I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all.

Hedgie🤗

_____________________________

The original source is Nature magazine: https://www.nature.com/articles/d41586-026-01100-y
Scientists invented a fake disease. AI told people it was real

[sdv]

Posted by Tintin (here)
This post deals with AI's inability to determine what's fake from real, in what could best be described as really very ironic indeed

It couldn't tell a joke either, I suspect.
________________________________________

Source: https://x.com/HedgieMarkets/status/2042430442448548273

🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks.
The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people.

ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher.

My Take
The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing.

I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all.

Hedgie🤗

_____________________________

The original source is Nature magazine: https://www.nature.com/articles/d41586-026-01100-y
Scientists invented a fake disease. AI told people it was real

I used to think that the most dangerous outcome for AI would be that most people would stop developing and using critical thinking skills. Now I am not that sure and see that the dangers go way beyond what I could imagine.

How can we get AI under control and still keep it as a useful tool?

[Jaak]

[ExomatrixTV]

• New Battles RAGE Across The Country Over Data Center Construction!:

Source: https://youtube.com/watch?v=KqOzexRjTf0

• We need to talk:

Source: https://youtube.com/watch?v=ktoWPIeVrzk

@DaveShap quote:

"This video has been up for 16 minutes, and I've already had to ban a few people for various levels of inciting violence or cheerleading violence. I am going to keep the comments open, but I encourage everyone to report violent comments, or comments that advocate or cheerlead more violence".

[ExomatrixTV]

• The AI Expert Who Thinks We've Already Lost — Dr Roman Yampolskiy

Source: https://youtube.com/watch?v=3I60uZEqXr0

00:00 Trailer 01:11 Why AI Safety Matters 05:11 Early Warnings & Risks 08:20 Exponential AI Progress 10:29 AI Survival Instincts 14:32 Can Nations Stop AI? 17:26 Ad: Quo 18:46 Why Safety May Be Impossible 25:19 Jobs, Meaning & Society 32:17 Best-Case AI Future 35:08 Ad: Qualia 36:47 AI Bias and Existential Risks 46:28 AI Warfare & Deepfakes 53:41 Ad: Hillsdale College 55:02 What Should We Do? 01:09:31 What's The One Thing We're Not Talking About?

[ExomatrixTV]

• The AI Backlash Has Reached a Tipping Point:

Source: https://youtube.com/watch?v=eDSaZveGttk

Data center pays no property taxes but if you want to build a barn on your own land now you have to pay more!