Cohere AI's LLM for Semantic Search in Python

James Briggs
Cohere AI's LLM for Semantic Search in Python
  • 1 hours
  • Beginner
  • Free
  • Online
In this video, we will learn how to use the Cohere Embed API endpoint to generate language embeddings using a large language model (LLM) and then index those embeddings in the Pinecone vector database for fast and scalable vector search.

Cohere is an AI company that allows us to use state-of-the-art large language models (LLMs) in NLP. The Cohere Embed endpoint we use in this video gives us access to models similar to other popular LLMs like OpenAI's GPT 3, particularly their recent offerings via OpenAI Embeddings like the text-embedding-ada-002 model. Pinecone is a vector database company allowing us to use state-of-the-art vector search through millions or even billions of data points. Both services together are a powerful and common combination for building semantic search, question-answering, advanced sentiment analysis, and other applications that rely on NLP and search over a large corpus of text data. 🌲 Pinecone docs: https://docs.pinecone.io/docs/cohere 🤖 70% Discount on the NLP With Transformers in Python course: https://bit.ly/nlp-transformers 🎨 AI Art: https://www.etsy.com/uk/shop/Intellig... 🎉 Subscribe for Article and Video Updates! https://jamescalam.medium.com/subscribe https://medium.com/@jamescalam/member... 👾 Discord: https://discord.gg/c5QtDB9RAP 00:00 Semantic search with Cohere LLM and Pinecone 00:45 Architecture overview 04:06 Getting code and prerequisites install 04:50 Cohere and Pinecone API keys 06:12 Initialize Cohere, get data, create embeddings 07:43 Creating Pinecone vector index 10:37 Querying with Cohere and Pinecone 12:56 Testing a few queries 14:35 Final notes

Sign up Course