Continuing our theme of running large language models (LLM) locally on your PC for freei; in this post we build upon the local AI chatbot from part 1, improving the responses by retrieving data to make it more knowledgeable. To do this, we use a technique called retrieval augmented generation (RAG), that adds data to the prompt before it goes to the model.
Conventional wisdom says RAG is a magic bullet that stops LLMs from going off topic, and eliminates hallucinations. But, is this really true?
Let’s explore further by diving into how RAG works, and run a simple demonstration.
Continue reading
Bits 'n Bytes 