Large language model predicts how to make inorganic compounds – Physics World

Large language model predicts how to make inorganic compounds – Physics World

Time Stamp: January 5, 2024 4:00 AM
Source Node: 2428662

Schematic diagram showing how the team trained the new model
The team trained the new model using chemical synthesis protocols extracted from 13 878 inorganic compounds, which they sourced from a collection of over four million scientific publications. Courtesy: Chinese Physics B

Researchers in China have fine-tuned an existing large language model (LLM) to create a system that can predict the steps required to synthesize an inorganic compound. Although the new model, dubbed MatChat, needs further refinement before it can be employed in the laboratory, its developers say it represents an important early endeavour in applying generative artificial intelligence in a scientific context.

LLMs are a type of artificial intelligence that extracts meanings from a sequence of text. To do this, they must first be trained on huge amounts of data. However, this training can be unsupervised, meaning that LLMs are said to be self-learning. They are also very flexible, able to perform tasks such as answering questions, writing text, translating languages and completing sentences. Well-known examples include Open AI’s GPT-3 model and its accessible spin-off, ChatGPT, which can already make predictions from a relatively small number of inputs or “prompts”.

To create an LLM that “understands” inorganic synthesis, researchers led by Miao Liu, a physicist at the Institute of Physics, Chinese Academy of Sciences, Beijing and founder of atomly.net, supplied a leading open-source LLM called LLaMA2-7B with additional scientific data. In doing so, Liu says, they “empower[ed] it to think akin to human cognition using a minimal dataset”. They trained the model using chemical synthesis protocols extracted from 13 878 inorganic compounds, which they sourced from a collection of over four million scientific publications compiled by Kristin Ceder and colleagues at the Lawrence Berkeley National Laboratory in the US.

“This training data allows the model to understand and respond to inorganic synthesis questions in the same way a knowledgeable expert would,” Liu explains. “For example, when queried about synthesising LiMnO2, MatChat can provide a detailed response that includes reaction precursors, equations and relative references in the literature.”

A typical exchange, he says, might look something like this:

Question: how to synthesize LiMnO2?

Answer: LiMnO2 can be made from Li2CO3, and MnO2 via a solid-state reaction. The chemical reaction equation is (0.5 Li2CO3 + 1 MnO2 == 1 LiMnO2 + 0.5 CO2). The precursors are mixed, calcined at 800.0°C in air for 2.0h, mixed, calcined at 800.0°C in air for 2.0h. The detailed recipe can be found in the literature…

A new project idea

Liu got the idea for the MatChat project in August 2023, after he attended a conference organized by Intel on the subject of information technology and AI. “Although the meeting had nothing to do with science, I learnt a lot about trending topics in AI and its applications,” Liu says. “It inspired me to apply the LLM to synthesis recipe prediction.”

To make the project happen, Liu teamed up with a colleague, Zongguo Wang, and a PhD student, Fankai Xie. While Xie trained the model, Wang built the freely available online platform that enables it to interact with users.

“While MatChat might not be the ultimate solution for this type of application, our work represents one of the early endeavours to apply LLM in a scientific context,” Liu tells Physics World. “We hope that that our study will serve as a catalyst for the creation of similar AI tools across multiple fields.”

Looking forward, the researchers plan to refine MatChat’s capabilities by expanding its dataset and integrating computational and experimental data from their own extensive materials science database, atomly.net, as well as a forthcoming robotic autonomous laboratory for inorganic materials synthesis. “Leveraging these resources, we aim to continue developing advanced AI tools for this field,” Liu says.

The new AI model is detailed in Chinese Physics B, and appeared in preprint form on the arXiv around the same time as a preprint from researchers at Microsoft who demonstrated a similar feat using the popular ChatGPT4 LLM.

Time Stamp:

More from Physics World