# How AI knows things: novice vs power user, pretrained knowledge, three retrieval modes (/docs/prompting/how-ai-knows-things) --- type: Procedure title: "How AI knows things: novice vs power user, pretrained knowledge, three retrieval modes" description: "Concepts 1 to 3 of the book's prompting guide: what separates a novice prompt from a power-user prompt, where a model's knowledge is strong, sparse or absent, and how to tell whether an answer came from memory, a web search or deep research, with the phrasings that steer each mode and the three lines that make web search reliable." status: stable order: 2 generated: { by: "process:claude-code", at: 2026-09-03T08:15:00Z } verified: - { by: "human:rrizwan98", at: 2026-09-03T08:20:00Z } sources: - id: af-03 title: "AI Prompting 2026: 13 Concepts, 80% of Real Use. The AI Agent Factory, Panaversity, 2026" resource: https://agentfactory.panaversity.org/docs/ai-prompting-2026 ksor: audience: [public] owner: human:rrizwan98 approval: { by: "human:rrizwan98", at: 2026-09-03T08:20:00Z } trust_tier: human-reviewed build_id: sha256:4d4bff09bd0a1dcff0a0a50f20f7ef514fc471e2b1f9e81e0e7b6150b93c0793 dirty: true --- Once you understand what is actually happening when you ask AI a question, you stop being surprised by the failures. [^af-03] ## Concept 1: novice vs power user Watch what changes between the two prompts. The question is the same; the briefing is not. [^af-03] - **Buying a car.** Novice: "which car is best?" Power user: uploads spec sheets, dealer quotes and insurance plans, then asks "what are the trade-offs? Read everything and think hard." - **Self-review at work.** Novice: "write a self-review for my boss." Power user: uploads a screenshot of their project tracker, recent project docs and a voice memo of notes, then asks for a draft. - **Critiquing a business idea.** Novice: "I have a great business idea, mobile tie-dyeing, critique it." That is sycophancy bait; the AI will mostly applaud. Power user: "Analyze objectively. Use this rubric: is there a problem worth solving, is there a market, is there a competitive advantage?" The AI scored that idea 8 out of 100 and explained why. - **Writing a blog post.** Novice: "write a blog post about the BlackBerry." Result: slop. **Slop** is the term of art for AI output that is fluent on the surface and empty underneath: grammatically clean, faintly Wikipedian, full of phrases like "in today's fast-paced world", and saying nothing a reader would remember an hour later. It is what AI produces by default when you give it no context and no constraints. Power user: outline first, critique the outline, expand each heading into bullets, critique the bullets, only then ask for prose. The mental model that ties these together: **AI is like a really smart fresh college grad. Highly motivated. Doesn't know much about you yet.** Brief them like one. Would a new colleague have enough information to do this job well? If not, give them more. [^af-03] ## Concept 2: pretrained knowledge AI did not learn by experiencing the world. It has no body, no senses, no time spent moving around in it. It learned by reading text about the world: massive amounts of internet text, forum threads, Wikipedia, books, news, research papers, blogs. Frequency in training data is roughly equal to reliability of the answer. [^af-03] - **Strong:** cooking, celebrity gossip, common medical advice, the top 1000 movies, popular programming languages, what is on the Voyager 1 record, why cats stare at walls. - **Sparse:** quasars, Cantonese (under 0.1% of internet text), regional history, niche professional knowledge. - **Absent:** your company's secret data, your private calendar, anything published after the model's knowledge cutoff date, anything someone never put on the public internet. Two practical consequences. **Don't waste time fixing typos:** AI was trained on internet text, which is full of them, and misspelling "definately" will not change the answer. **Watch for absorbed errors:** AI also absorbed misconceptions and outdated information from those same sources. A confidently wrong forum post becomes confidently wrong in the model. Check anything important against a primary source. [^af-03] A quick mental test before you trust a pretrained answer: [^af-03] | Question type | How well represented in training data? | Trust level | | --- | --- | --- | | "How do I make a roux?" | Cooking is one of the most discussed topics on the internet | High | | "Plot of a top-1000 movie" | Reviewed and re-reviewed thousands of times | High | | "History of an obscure village" | Possibly one Wikipedia paragraph, or none | Low; verify against a primary source | | "Recent regulatory change in my industry" | Almost certainly after the knowledge cutoff | Trust nothing without web search | | "What did our company decide last quarter?" | Not in the training data at all | Trust nothing; the model is guessing | This is not a rule to memorize. It is the same instinct you apply to any other source: "how would this person know that?" Apply it to AI too. Confidence is not a signal of correctness; the first place to look for broken reasoning is in confident-sounding answers about topics where the training data was thin or contested. [^af-03] The book's example: a reader asked for the rules of a regional folk game played in their grandmother's village. The AI confidently produced three paragraphs of rules. The grandmother said they were almost entirely wrong: the AI had blended similar games from other regions because the specific game was barely on the internet. The AI did not lie; it generalized from sparse data. The reader's mistake was assuming confidence equaled accuracy. [^af-03] ## Concept 3: the three retrieval modes When you ask a question, modern tools quietly choose how to answer: from pretrained knowledge alone, by firing a web search and reading a few pages, or by running deep research, several minutes across dozens of sources ending in a structured report. Know which mode is firing, because each has different strengths and failure modes. [^af-03] - **Pretrained answers fine:** "why do cats stare at walls", "what's on the Voyager 1 record", "summarize the plot of Hamlet". These do not change week to week. - **Web search rescues a stale model:** anything that went viral after the cutoff is invisible without it. A meme, a regulation, a product launch. - **Web search going wrong:** asked "where to run in Henderson, Nevada", the AI cited a 20-year-old page and recommended a school no longer open to the public. Web search does not check whether sources are current. - **Deep research worth the wait:** "plan a Halloween haunted house in our neighborhood, including permits, fire safety, and noise ordinances." The AI proposes a research plan, runs many parallel searches, summarizes, decides what to dig into next, and produces a multi-section report with checklists. That is closer to handing the work to a junior researcher for an hour. ### How web search actually works, and why it misreads pages A **search-and-retrieval layer** issues the searches, scans the result list, pulls the most relevant pages and reduces each to a short passage or summary; often that layer is a separate, smaller model. Only the reduced version flows to the model that talks to you. So it frequently does not read the original page; it reads a condensed version, and translation layers lose nuance. [^af-03] Two fixes. **Tell the AI which kinds of sources to use:** instead of "are vaccines safe", try "use the World Health Organization, the FDA, the European Medicines Agency, and peer-reviewed studies. Do not use forums or personal blogs." Source quality is a knob you can turn; default settings cite popular sources first. **Ask the AI to quote the source:** "For each claim, quote the exact sentence from the source page that supports it." This forces the retrieval layer to surface original wording, which catches a lot of summary-layer drift. [^af-03] ### Steering the mode with your wording You usually do not pick a mode by clicking a button; the AI picks based on your prompt. But you can steer: [^af-03] | Phrasing pattern | What it usually triggers | | --- | --- | | "What is X" / "Summarize Y" | Pretrained only | | "What's the latest on X" / "Today" / "This week" / a specific city | Web search | | "Research X thoroughly", "produce a report with citations", "use these source types" | Deep research (in tools that have it; otherwise extended web search) | | Attaching files | Stays pretrained for the files; may search the web for context if the prompt asks for current info | ### AI vs Google They are not the same tool. Use Google for quick scans, navigating to a specific known site, or buying a thing. Use AI when you need synthesis: pros and cons, multi-source comparison, a written-out analysis. The choice depends on whether you want a link or an answer. [^af-03] | Task | Better with Google | Better with AI | | --- | --- | --- | | "Find the official IRS page for form 1040." | Yes. You want a specific known site | No | | "Compare three diabetes medications and what the recent evidence says." | Slower. You'll read 8 tabs | Faster. AI synthesizes the evidence in one place | | "Buy a replacement charger for a 2018 ThinkPad." | Yes. You want a product link | No | | "Plan a 4-day Lisbon trip with a 6-year-old, no museums." | Slow. You'll juggle blogs and reviews | Fast. AI integrates constraints | | "What's the weather tomorrow?" | Either | Either | | "Why are my tomato plant leaves yellowing?" | OK. Multiple gardening sites | Better with a photo attached | If your question is "where is X", reach for Google. If your question is "given all this, what should I think", reach for AI. [^af-03] ### Three lines that make web search reliable When you do want web search, three small habits raise the quality: [^af-03] 1. **Name the sources you trust.** "Use the WHO, the FDA, and peer-reviewed studies, not forums." 2. **Ask for citations inline.** "Cite the source after each claim." 3. **Ask the AI to flag what it could not verify.** "If a claim cannot be supported by the cited sources, mark it 'unverified'." Pasted into any web-search prompt, these three lines cut down the most common failure: the AI quietly synthesizing across sources into a confident sentence that no single source supports. [^af-03] Continue to [context is the whole game](context-is-the-whole-game.md). [^af-03]: AI Prompting 2026: 13 Concepts, 80% of Real Use, Part 1 "How AI knows things", Concepts 1, 2 and 3. The AI Agent Factory, Panaversity, 2026.