Sentence Similarity and Semantic Search using free Huggingface Embedding API
Sentence similarity involves determining the likeness between two texts. The idea behind semantic search is to embed all entries in your corpus, whether sentences, paragraphs, or documents, into a vector space. The query is embedded into the same vector space at search time, and the closest embeddings from your corpus are found. Some applications of sentence similarity include question answering, passage retrieval, paraphrase matching, duplicate question retrieval, and semantic search. In this article, we will explore semantic search. The application uses sentence similarity to implement a document search on a Medium blog article. The input to the application will be a question/sentence, and the output will be a set of sentences containing semantically similar content to the input sentence. The Sentence Transformers library The Sentence Transformers library is open-source for creating state-of-the-art embeddings from text and computing sentence similarity. Hugging Face offer...