Everyone is using AI nowadays and that includes Researchers and Scientists. While AI has not done most of its work till now, things are changing. Imagine if you have an automated AI Scientist that does idea generation, conducts experiments, writes papers and then reviews them; all for 15 dollars. Sounds interesting right!!
Highlights:
- Researchers at Sakana AI develop a fully automated AI system that writes research papers for 15 dollars.
- It can save a lot of time for scientists and lead to faster advancements in various fields.
- The model still has limitations and the results published are not completely accurate.
How does the new Sakura’s AI Scientist work?
The “AI Scientist” is developed by Researchers from the University of British Columbia, and the University of Oxford, Sakana AI. It can generate new research ideas, perform experiments, and even visualize results.
“Today, we’re excited to introduce The AI Scientist, the first comprehensive system for fully automatic scientific discovery, enabling Foundation Models such as Large Language Models (LLMs) to perform research independently.”
That means it can write a full research paper for just $15 per paper. Look at the diagram to see how it works (from Sakana AI’s website):
The AI scientist will be first presented with an existing code template based on which it starts the brainstorming process. The AI is free to explore in all possible research directions. At this step, it is possible to present the model with a latex template that contains all the necessary style files and section headers based on the requirements.
Later the AI is presented with a semantic scholar that checks if the idea generated is novel or not. It scores the idea and proceeds to the experimentation phase if the score is good enough. Otherwise, the idea is archived.
The AI scientist then starts conducting all the experiments and notes down the results. It even produces plots and graphs that visualize the outcomes of the experiments conducted.
Then, it writes up all the information gathered in the style of a standard machine learning conference in LaTeX. It even finds references to cite by using Semantic Scholar. After completing the research paper, an automated LLM reviewer evaluates the paper. This reviewer is said to be capable of evaluating generated papers with near-human accuracy.
Researchers at Sakana AI have demonstrated how AI scientist works by producing 3 papers on key machine learning topics: Diffusion Modeling, Language modelling and Grokking.
Limitations and Problems
Problems that are yet to be resolved in this model include: not being able to fix visual issues with the paper and read plots properly.
Sometimes, the AI scientist tends to generate unreadable figures. The researchers have even encountered several issues with the AI scientist that we also have with regular LLMs like being unable to compare the magnitude of two numbers. This leads to the model making critical errors especially when writing the results.
There are also many other problems like using the potential of the model for conducting many unethical researches like creating dangerous computer viruses. We must understand that this is the first step towards enhancing the role of AI in scientific research.
The paper generated by the AI Scientist has been labelled “weak accept” by a machine learning conference. This means the papers produced can not still meet the quality of research done by humans manually.
Does this mean it has no use? No, the model is a significant milestone in the AI research field. The AI scientist helps save a lot of time by conducting all the research, writing the results in a LaTeX format and generating plots. This is extremely helpful in fields where time is critical, such as medical research or environmental science.
Conclusion
This technology has the potential to democratize access to academic publishing by automating the process of generating research papers and making them available at a reasonable cost. The makers believe that the AI scientist will make a great companion to human scientists. For more such studies, check out AI Articles on the latest developments.
