Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notes

In a groundbreaking study by a team of researchers, a new large language model named Asclepius has been developed, specifically tailored for handling patients' clinical notes. The challenge of limited accessibility and usability of these notes due to strict privacy regulations has been successfully addressed by creating synthetic clinical notes. These synthetic notes were created using publicly available case reports extracted from biomedical literature, which were then used to train Asclepius.

The performance of Asclepius was evaluated using real clinical notes from diverse sources and benchmarked against several other large language models, including GPT-3.5-turbo. Interestingly, Asclepius, trained on synthetic notes, demonstrated performance closely aligned with that of Asclepius-R, a variant trained on real clinical notes. This suggests that synthetic clinical notes can serve as viable substitutes for real ones when constructing high-performing clinical language models.

The success of Asclepius has been validated not only by GPT-4 but by medical professionals as well. While GPT-4 evaluated the model's performance based on helpfulness and fluency, medical professionals prioritized accuracy, relevancy, and completeness. The detailed evaluations conducted by both parties supported the effectiveness of synthetic clinical notes.

This innovative approach opens new doors for future research and development in the field of clinical language models. All resources, including weights, codes, and data used in the development of Asclepius, have been made publicly accessible, further encouraging research in this area.

Read the whole article here: http://arxiv.org/abs/2309.00237v1