After working with Large Language Models (LLMs), I was amazed by their reasoning power, but what really excited me was making LLM even more useful by connecting them to real-world data and tools. Before I learned about MCP, I tried to connect LLMs to real-world external tools such as internet search, music playback, and email sending. Eventually I managed to build a smart assistant that could handle these tasks, however the implementation was really messy and fragile. The code relied heavily on conditional prompting, for example:<\/p>\n\n\n\n
I then had to sanitize the LLM\u2019s output, execute the generated code in Python, capture the result, and pass it back to the LLM again to produce the final response. While this approach technically worked, it was error-prone, hard to maintain, and often broke in edge cases. Adding any new functionality was a huge pain, as it required rewriting prompts, adding more conditions, and increasing the overall complexity of the system.<\/p>\n\n\n\n When I learned about MCP, I literally said, \u201cWow<\/strong>.\u201d<\/em> that’s what I really was looking for. Instead of brittle prompt engineering and manual code execution, MCP offered a clear separation of concerns, better reliability, and far easier extensibility. What previously felt like a hack suddenly became an elegant and scalable architecture.<\/p>\n\n\n\n Regarding RAG, my early experiments was enriching models with the right context to produce more accurate and context-aware responses. I found it to be a powerful approach for building smart agents that can answer questions within a specific domain, especially when grounded in private or proprietary databases. By retrieving relevant information at query time and feeding it to the model, RAG enables precise and up-to-date answers without requiring model retraining<\/strong>. This makes it particularly well suited for use cases such as internal knowledge assistants, documentation search, and domain-specific Q&A systems.<\/p>\n\n\n\n Indeed, LLMs are like geniuses locked in a room with no internet or tools. They can think and explain, but they can\u2019t reach out and do<\/em> anything<\/strong>. If you ask:<\/p>\n\n\n\n \u201cWhat\u2019s the temperature in New York right now?\u201d<\/p>\n<\/blockquote>\n\n\n\n LLM will hallucinate or guess, because it has no access to real data. And if you say:<\/p>\n\n\n\n \u201cPlay my favorite song\u201d<\/p>\n<\/blockquote>\n\n\n\n It\u2019ll describe how to play it\u2026 but won\u2019t touch your speakers. To make AI useful<\/strong>, we need to connect these models to tools<\/strong>, data<\/strong>, and actions<\/strong> safely.<\/p>\n\n\n\n In this post, I\u2019ll walk you through how to set up a RAG pipeline and an AI agent using a local LLM with Ollama and LangChain.<\/p>\n\n\n\n LangChain <\/strong>is an open-source framework designed to help developers build applications powered by Large Language Models (LLMs), such as GPT, Claude, or Llama. Instead of using an LLM as a simple text generator, LangChain <\/strong>allow us to connect In short,<\/strong> LangChain helps you turn a language model into an intelligent agent<\/em> that can think, use tools, and take actions autonomously.<\/strong><\/p>\n<\/blockquote>\n\n\n\n One of great features of the LangChain is providing the easy RAG setup. It enables our model fetch context<\/strong> from external data (documents, databases, etc.) before answering.<\/p>\n\n\n\n Here\u2019s a minimal RAG example using LangChain<\/strong>. Assume we have dataset of weather information and we want AI answer user’s question based on the dataset facts. In simple words, we should feed LLM with our dataset and ask it to answer the questions based on that :<\/p>\n\n\n\n Output:<\/p>\n\n\n\n \u201cTomorrow is raining in Berlin.\u201d<\/p>\n<\/blockquote>\n\n\n\n That\u2019s RAG in action. Giving our LLM fresh memory without retraining. It starts by defining a small knowledge base of text documents and splits them into manageable chunks. Each chunk is then converted into a numerical representation <\/strong>(embeddings<\/strong>) using the Now we understand that RAG gives the model knowledge<\/em>, but not arm to take actions like playing music or sending email. To let it perform tasks, LangChain <\/strong>introduces Agents<\/strong> that can reason<\/em> and then act<\/em> using tools.<\/p>\n\n\n\n Example: a simple agent that can use a calculator, Wikipedia, and a method that reverse given text.<\/p>\n\n\n\n Output<\/strong>:<\/p>\n\n\n\n \u201cAlbert Einstein was born in 1879. 1879 \u00d7 2 = 3758. And the reverse of the result is 8573.\u201d<\/p>\n<\/blockquote>\n\n\n\n The agent looked up Einstein\u2019s birth year via Wikipedia, used the calculator tool and combined both into a reasoning chain, finally using reverse method it reversed the result . That\u2019s a small taste of agentic reasoning<\/strong>. The model doesn\u2019t just answer, it decides how<\/em> to answer. First it retrieves related information from Wikipedia, perform calculation on the birthday, then he decide to use reverse tool to generate final result.<\/p>\n\n\n\n
\u201cIf the user asks to search for something on the internet, generate a function call like result = search_internet(query)<\/code> with no explanation, output only Python code inside a code block.\u201d<\/em><\/p>\n\n\n\n\n
\n
LangChain<\/h2>\n\n\n\n
the model to real data sources, external APIs, or even custom Python functions. This gives the LLM the ability to reason, plan, and act through tools. <\/p>\n\n\n\n\n
Retrieval-Augmented Generation (RAG)<\/h2>\n\n\n\n
# pip install langchain langchain-ollama faiss-cpu\n\nfrom langchain_ollama import ChatOllama\nfrom langchain.text_splitter import CharacterTextSplitter\nfrom langchain.vectorstores import FAISS\nfrom langchain.embeddings import OllamaEmbeddings\nfrom langchain.chains import RetrievalQA\nfrom langchain.docstore.document import Document\n\n# 1. Create your \"knowledge base\"\ndocs = [\n Document(page_content=\"Tomorrow it is raining all across Germany.\"),\n Document(page_content=\"It's windy in London Today.\")\n]\n\nsplitter = CharacterTextSplitter(chunk_size=200)\nchunks = splitter.split_documents(docs)\n\n# 2. Build a retriever (index)\nembeddings = OllamaEmbeddings(model=\"mxbai-embed-large\")\nvectorstore = FAISS.from_documents(chunks, embeddings)\n\n# 3. Create a retrieval-based QA chain\nqa = RetrievalQA.from_chain_type(\n llm=ChatOllama(model=\"llama3\"),\n retriever=vectorstore.as_retriever()\n)\n\n# 4. Ask something beyond the model\u2019s memory\nquery = \"What's the weather like tomorrow in Berlin?\"\nprint(qa.run(query))\n<\/code><\/pre>\n\n\n\n\n
OllamaEmbeddings<\/code> <\/strong>model. These embeddings are stored in a FAISS <\/strong>vector database, which allows efficient similarity searches. When a user asks a question, the retriever searches for the most relevant text chunks based on their embeddings and passes them, along with the query, to the local language model (llama3<\/strong><\/code>). The model then generates a final answer grounded in the retrieved context, effectively extending its knowledge beyond its built-in training data.<\/p>\n\n\n\nAI Agent<\/h2>\n\n\n\n
# pip install langchain langchain-ollama wikipedia\n\nfrom langchain.agents import initialize_agent, load_tools, AgentType, Tool\nfrom langchain_ollama import ChatOllama\n\n# === Step 1: Initialize the local Ollama model ===\nllm = ChatOllama(model=\"llama3\", temperature=0)\n\n# === Step 2: Load built-in tools ===\ntools = load_tools([\"wikipedia\", \"llm-math\"], llm=llm)\n\n# === Step 3: Define a custom tool ===\ndef reverse_text(text: str) -> str:\n \"\"\"\n A simple custom tool that reverses a string.\n You can replace this with any real logic you want \u2014 e.g., playing music, reading files, etc.\n \"\"\"\n return text[::-1]\n\ncustom_tool = Tool(\n name=\"ReverseText\",\n func=reverse_text,\n description=\"Reverses the given text string.\"\n)\n\n# === Step 4: Add the custom tool to the list ===\ntools.append(custom_tool)\n\n# === Step 5: Initialize the agent ===\nagent = initialize_agent(\n tools,\n llm,\n agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n verbose=True\n)\n\n# === Step 6: Run an example query ===\nresponse = agent.invoke({\n \"input\": \"Find the year Einstein was born, multiply it by 2, and then reverse that number as text.\"\n})\n\nprint(\"\\nFinal Answer:\")\nprint(response[\"output\"])\n<\/code><\/pre>\n\n\n\n\n
MCP (Model Context Protocol)<\/h2>\n\n\n\n