D 2018

Implementation Notes for the Soft Cosine Measure

NOVOTNÝ, Vít

Základní údaje

Originální název

Implementation Notes for the Soft Cosine Measure

Autoři

NOVOTNÝ, Vít

Vydání

Torino, Italy, Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM '18), od s. 1639-1642, 4 s. 2018

Nakladatel

Association for Computing Machinery

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Stát vydavatele

Itálie

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL, URL

Označené pro přenos do RIV

Ano

Kód RIV

RIV/00216224:14330/18:00101853

Organizace

Fakulta informatiky – Masarykova univerzita – Repozitář

ISBN

978-1-4503-6014-2

DOI

https://doi.org/10.1145/3269206.3269317

UT WoS

000455712300190

EID Scopus

2-s2.0-85058046690

Klíčová slova anglicky

Vector Space Model; computational complexity; similarity measure

Návaznosti

MUNI/A/1038/2017, interní kód Repo. MUNI/A/1213/2017, interní kód Repo. TD03000295, projekt VaV.
Změněno: 26. 4. 2022 03:08, RNDr. Daniel Jakubík

Anotace

V originále

The standard bag-of-words vector space model (VSM) is efficient, and ubiquitous in information retrieval, but it underestimates the similarity of documents with the same meaning, but different terminology. To overcome this limitation, Sidorov et al. proposed the Soft Cosine Measure (SCM) that incorporates term similarity relations. Charlet and Damnati showed that the SCM is highly effective in question answering (QA) systems. However, the orthonormalization algorithm proposed by Sidorov et al. has an impractical time complexity of O(n^4), where n is the size of the vocabulary. In this paper, we prove a tighter lower worst-case time complexity bound of O(n^3). We also present an algorithm for computing the similarity between documents and we show that its worst-case time complexity is O(1) given realistic conditions. Lastly, we describe implementation in general-purpose vector databases such as Annoy, and Faiss and in the inverted indices of text search engines such as Apache Lucene, and ElasticSearch. Our results enable the deployment of the SCM in real-world information retrieval systems.
Zobrazeno: 6. 5. 2026 16:25