Inproceedings,

Language Models as Context-sensitive Word Search Engines

, , , and .
Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022), page 39--45. Dublin, Ireland, Association for Computational Linguistics, (May 2022)
DOI: 10.18653/v1/2022.in2writing-1.5

Abstract

Context-sensitive word search engines are writing assistants that support word choice, phrasing, and idiomatic language use by indexing large-scale n-gram collections and implementing a wildcard search. However, search results become unreliable with increasing context size (e.g., n\textgreater=5), when observations become sparse. This paper proposes two strategies for word search with larger n, based on masked and conditional language modeling. We build such search engines using BERT and BART and compare their capabilities in answering English context queries with those of the n-gram-based word search engine Netspeak. Our proposed strategies score within 5 percentage points MRR of n-gram collections while answering up to 5 times as many queries.

Tags

    Users

    • @scadsfct

    Comments and Reviews