Artikel in einem Konferenzbericht,

Large Language Models for Scientific Question Answering: An Extensive Analysis of the SciQA Benchmark

, , , , , , und .
ESWC 2024, 14664, Seite 199--217. Cham, Springer, (2024)

Zusammenfassung

The SciQA benchmark for scientific question answering aims to represent a challenging task for next-generation question-answering systems on which vanilla large language models fail. In this article, we provide an analysis of the performance of language models on this benchmark including prompting and fine-tuning techniques to adapt them to the SciQA task. We show that both fine-tuning and prompting techniques with intelligent few-shot selection allow us to obtain excellent results on the SciQA benchmark. We discuss the valuable lessons and common error categories, and outline their implications on how to optimise large language models for question answering over knowledge graphs.

Tags

Nutzer

  • @scadsfct

Kommentare und Rezensionen