Scientists are doing a lot of research in the field of AI, but now AI itself has become a scientist. It can do all the scientific stuff from start to finish. To prove that, I recently learned about this development that an ‘AI Scientist’ has written a science paper that has successfully passed the peer-review process.
Sakana’s AI Scientist Passes Peer Review
Let’s start with what is Sakana’s AI Scientist if you don’t know anything about it. AI Scientist is an innovative system designed to automate the entire scientific research process. This AI-driven framework independently generates research ideas, designs and conducts experiments, and analyzes results. It was able to write research papers for 15 dollars.
Recently, the company collaborated with a workshop at the ICLR (International Conference on Learning Representations) 2025. They conducted an experiment to test the capabilities of The AI-Scientist-v2. This is the new version of the AI scientist that has been not released to the public yet.
They submitted a total of 43 papers to the workshop with 3 of them being AI-generated. Note that they also informed the reviewers that some submissions might be written by AI.
The goal was to see if these AI-generated research papers could meet the rigorous standards of academic peer review.
So, when the process started, all eyes were on this process. 2 of the papers failed.
But the 3rd paper titled “Compositional Regularization: Unexpected Obstacles in Enhancing Neural Network Generalization” received favorable reviews. It got a rating of 6.33.
This paper reported a negative result that The AI Scientist encountered while trying to innovate novel regularization methods for training neural networks that can improve their compositional generalization.
The AI Scientist-v2 was just given the topic by a human, it did the rest of the things. It autonomously formulated a scientific hypothesis, designed experiments, wrote code, conducted research, and analyzed data. It even wrote the complete manuscript.
In the end, it surpassed the acceptance threshold. This means the AI-generated paper is equally good as human-authored research.
The best part was that it was all done with being transparent and not in some secret way. So, after the peer review, the paper was withdrawn.
However, the paper was accepted only at the workshop level, not at the main conference. Workshops generally have higher acceptance rates compared to main conferences.
The Future of LLMs in Research
According to Sakana, this was the first time such a thing had happened: An AI-generated paper was reviewed and passed.
They also mentioned that their system is mostly off on the best LLMs today. So, this performance shows how good the current LLMs have become.
The development of systems like Sakana AI’s AI Scientist highlights the advancements in large language models and their reasoning capabilities. This AI system autonomously performed tasks traditionally handled by human researchers, including hypothesis generation, experiment design, and data analysis.
We have seen OpenAI’s o-series reasoning models getting better day-by-day. However, they are general-purpose reasoning engines designed to tackle a broad spectrum of complex tasks. So, AI is becoming more and more human-like in some way.
Takeaways
This achievement is for sure a significant step forward in integrating AI into the scientific process. I also want to mention that it was not a perfect job as reviewers did point out areas for improvement. Sakana’s internal review found that The AI Scientist made some citation errors. They also were not completely satisfied with the paper in the end and will improve their system accordingly.
Still, the success of The AI Scientist-v2 opens the possibility of AI systems capable of assisting real scientists and accelerating discoveries.
