Week 2 of 6
Every AI agent runs on a Large Language Model. Understanding how it works at a basic level is the difference between using AI and truly building with it.
Here is the simplest accurate explanation: an LLM is a next-token predictor. It was trained on a massive amount of text and learned one thing extremely well: given everything that came before, what word is most likely to come next? Scaled to hundreds of billions of parameters trained on trillions of words, that becomes something that looks a lot like reasoning and knowledge.
An LLM does not know facts the way you know your own name. It learned patterns so well that it can generate text that looks like knowledge. Most of the time that is indistinguishable from real knowledge. Sometimes it is not — and that is hallucination.
When an agent uses an LLM to reason, it is asking: given my goal, what I know, and the tools available to me — what should I do next? The LLM generates the next step. Then the agent acts on it. Then feeds the result back in. The loop from Week 1, powered by a text predictor.
You have probably seen this: you ask an AI a question, it answers confidently, and the answer is completely wrong. That is hallucination — and understanding why it happens changes how you use AI forever.
LLMs generate plausible text, not verified facts. The model was trained to predict what a reasonable response looks like — not to retrieve truth from a database. When it does not know the answer, it does not say so. It generates what an answer would look like. Confidently.
| Search Engine | LLM |
|---|---|
| Retrieves existing documents | Generates new text |
| Can say no results found | Always produces an answer |
| Links back to sources | May invent sources |
| Factually bounded | Creatively unbounded |
Never trust an LLM for facts you cannot verify — especially dates, names, statistics, and citations. Treat it as a very smart collaborator who sometimes makes things up. Your job is to add guardrails: tools that verify, sources that ground, humans who check.
The way you write instructions to an LLM changes everything. The same question phrased differently can produce wildly different results. This is prompt engineering — one of the most valuable skills in AI right now.
Weak: "Explain machine learning."
Strong: "You are a high school teacher. Explain machine learning to a student who knows basic algebra but has never coded. Use one real-world analogy. Keep it under 100 words."
You just learned why AI hallucinates. Now the question that matters in the real world: when is being wrong merely annoying, and when is it harmful? A hallucination is the AI confidently wrong (a mechanism). Misinformation is when a wrong answer causes harm because someone acted on it (an outcome). "The AI invented a fun-fact for my essay" and "the AI invented a medication dosage someone followed" are the same hallucination — very different harm. The skill isn't just noticing AI is wrong; it's fact-checking before you act, especially when the stakes are high.
Open Claude.ai or ChatGPT. Complete all three rounds. Write your exact prompt in each box, then paste the AI response. What made the winning prompts work?
def greet(name): print("Hello " + name) greet()Go Deeper — Optional
Run in Google Colab — no setup, no install. Interactive Python exercises that reinforce every concept from this lesson.
Coming up — Week 3
Tools and Memory
Why bare LLMs are limited and how tools and memory transform them.