KeyBERT

KeyBERT is a Python library that extracts keywords and phrases by comparing their meaning with the document they come from. It's for developers and researchers who want to add keyword extraction to text processing projects without training a model from scratch. The library is open source under the MIT license.

Its embedding approach ranks words and phrases by how closely they match the document as a whole. You can choose single words or longer phrases, and process multiple documents together. The results include similarity scores, so applications can use the rankings rather than treating every extracted term equally.

For documents that produce several closely related terms, Max Sum Distance and Maximal Marginal Relevance (MMR) help reduce repetition in the results. MMR lets you adjust how much variety you want in the selected keywords.

KeyBERT supports Sentence-Transformers, Flair, spaCy, Gensim and Universal Sentence Encoder. It also supports Model2Vec for a lighter setup that doesn't require PyTorch. The Python library can use local embedding models; its optional KeyLLM integration can send text to OpenAI's ChatGPT models for keyword extraction.

KeyLLM can also reuse keywords across sufficiently similar documents, reducing the need to ask the LLM about every document separately. A similarity threshold controls which documents receive the same keywords.