Evaluation of large language models for discovery of gene set function

In a groundbreaking new study, a team of researchers have evaluated the potential of OpenAI's GPT-4, a Large Language Model (LLM), in the field of functional genomics. The focus of the study is gene set analysis, a critical aspect of functional genomics that relies heavily on manually curated databases of gene functions. These databases, while useful, are incomplete and lack the ability to adapt to different biological contexts.

The researchers created a GPT-4 pipeline to label gene sets with names that summarize their consensus functions. The names were supported by analysis text and citations. The performance of GPT-4 was benchmarked against named gene sets in the Gene Ontology, a database of gene functions. The results were impressive: GPT-4 generated very similar names in 50% of cases, while in most remaining cases it recovered the name of a more general concept.

In gene sets discovered in 'omics data, GPT-4 names were found to be more informative than gene set enrichment. The supporting statements and citations generated by GPT-4 largely verified in human review. This suggests that GPT-4 has the ability to rapidly synthesize common gene functions, positioning LLMs as valuable assistants in functional genomics.

However, the use of LLMs in functional genomics is not without challenges. For instance, formulating appropriate prompts to guide the response of the model might be tricky. Also, while the model generates supporting statements and citations, it occasionally produces unverifiable statements, indicating the need for some form of fact-checking or reference validation.

In conclusion, this study demonstrates the potential of LLMs like GPT-4 in enhancing our understanding of gene function. As we continue to improve these models and address their limitations, we can expect to see more applications of LLMs in functional genomics and other areas of biomedical research.

Read the whole article here: http://arxiv.org/abs/2309.04019v1