# Safety, cost and checking: permissions, which model when, models checking models (/docs/prompting/safety-cost-and-checking) --- type: Procedure title: "Safety, cost and checking: permissions, which model when, models checking models" description: "Concepts 11 to 13 of the book's prompting guide: the plan-then-approve workflow and permission ladder for file-aware AI apps, the cost and speed stack, why no single model is best and how to keep tooling current, and the full recipe for making models from different families grade each other, with the single-model self-critique loop for everyday work." status: stable order: 7 generated: { by: "process:claude-code", at: 2026-09-03T08:15:00Z } verified: - { by: "human:rrizwan98", at: 2026-09-03T08:20:00Z } sources: - id: af-03 title: "AI Prompting 2026: 13 Concepts, 80% of Real Use. The AI Agent Factory, Panaversity, 2026" resource: https://agentfactory.panaversity.org/docs/ai-prompting-2026 - id: arena title: "Arena leaderboard: blind head-to-head model comparisons" resource: https://arena.ai/leaderboard ksor: audience: [public] owner: human:rrizwan98 approval: { by: "human:rrizwan98", at: 2026-09-03T08:20:00Z } trust_tier: human-reviewed build_id: sha256:4d4bff09bd0a1dcff0a0a50f20f7ef514fc471e2b1f9e81e0e7b6150b93c0793 dirty: true --- Three final concepts: how to safely give AI access to your files and permissions, how to pick the right tool for the job, and how to get an objective signal on quality when no human expert is in the room. [^af-03] ## Concept 11: AI desktop apps and permissions There is now a category of **AI desktop apps** that run on your computer and, with permission, find your files, read them, and act on them. What they can do that chat cannot: look through a messy folder of PDFs, propose a new organization, and execute it once you approve; pull together related files for a project and notice things on their own; read across a folder and summarize what you worked on last quarter. [^af-03] The workflow that makes this safe: [^af-03] 1. **Tell it the task.** "Reorganize this folder by client." 2. **Ask for a plan, not action.** The app proposes a list of file operations. 3. **Review and edit the plan.** Catch the rename you do not want before it happens. 4. **Only then approve execution.** Two facts most people learn the hard way: **deleted files often do not go to your recycle bin** when an AI app deletes them; and **edited files do not keep an edit history** unless you have version control. Until you have done this safely a few times, scope every permission request to the smallest folder needed. Do not approve full disk access for an app you have used twice. [^af-03] A consultant's `clients/` folder had grown to 240 PDFs over four years. She told the app: "Look through `clients/`. Propose an organization scheme. Do not move any files yet. Show me the proposed scheme as a tree." It produced one folder per client with sub-folders, plus a flagged list of 18 files it could not classify. She edited the proposal, then approved. About fifteen minutes, for a job that had sat on her someday list for three years. [^af-03] | Comfort level | What to allow | What to keep saying no to | | --- | --- | --- | | First sessions | Read-only access to a single small folder | Anything that writes, deletes or renames | | After 2-3 successful runs | Read and write inside one specific folder | Broader directories like the desktop or documents root | | After a clean week | Read across a project tree, write inside a scoped subfolder | Anything outside that project | | Trusted | Tool-specific permissions ("rename PDFs in this folder") | Open-ended "do whatever you need" | The principle: scope grows with track record, not with how much you trust the company that built the tool. Trust is earned by behaviour in your specific workflow. [^af-03] This is the same ladder the [loop and harness](../agent/loop-and-harness.md) page applies to a worker. ## Concept 12: cost, speed and which model to use when A simple stack, from fastest and cheapest to slowest and dearest: **text** (seconds), **speech** (seconds, per minute of audio), **images** (tens of seconds, no early stop because the whole image generates at once), **video** (minutes per generation, and iteration is painful because each round is slow and expensive), and **deep research** (minutes, but it synthesizes dozens of sources for you). Cost is barely a constraint at the entry level: the major chatbots all offer free access that handles the prompts in this guide comfortably. Two implications: iteration cost shapes what you do, so invest more in the prompt up front for images and video and use a text AI to write it; and costs are trending down. [^af-03] **Which model for which task?** AI is **jagged**: different models are good at different things, and the leader changes every few months. There is no single best model. Two habits: **try the same prompt in 2 to 3 models routinely**, and **don't marry one tool**; switching is free, you paste the prompt in a different tab. A rough mid-2026 snapshot, a starting point and not a verdict: [^af-03] | Tool | Tends to be strong at | Tends to be weaker at | | --- | --- | --- | | Claude | Reasoning on hard prompts, long-document understanding, SVG and diagrams, code, careful writing voice, structured analysis | In-product photo-realistic image generation is less central | | ChatGPT | Top-ranked in-product image generation, voice mode, conversational range, broad task coverage | Sometimes verbose; can over-format with lists and headings | | Gemini | Fast web search and source synthesis, deep research with rich output, strong image generation, Google Workspace integration | Tone can feel clipped; some responses lean short | | Meta AI | Embedded in WhatsApp, Instagram and Messenger, free, competitive multimodal reasoning since Muse Spark (April 2026) | Coding and long-horizon agents lag; smaller ecosystem; rate-limited under heavy use; inputs may be used for training by default | | DeepSeek | Open-source weights, self-hostable, low cost, 1M-token context by default | Chat polish and consumer ecosystem trail the big three | Inside a single brand there is a **model ladder**: a fast, cheap default for everyday work, a slower reasoning level above it, and a heavy flagship for the hardest tasks. A batch of short replies on the flagship spends budget for nothing; your hardest analysis on the cheapest rung wastes the analysis. The rung names rotate, so read the picker rather than your memory. [^af-03] **The leaderboard to bookmark** is Arena, where users vote in blind head-to-head comparisons, with separate boards for text, code, vision, document, image generation, image edit, search and video. Check it once a month; leaders rotate quickly. Two caveats: leaderboards reward conversational charm more than careful work on long documents, and they sample tasks that voters find interesting, which is not always your task. [^af-03] [^arena] Three habits that compound: **have at least two tabs open**, a primary and a backup, and paste the same prompt into the backup when the primary does not feel right; **keep a prompt scratchpad**, a note file of prompts that produced unusually good results; **notice when the model is wrong**, as data, not scolding. Once a month, glance at the leaderboards and run one task you do regularly through three tools; use the winner until next month. [^af-03] ## Concept 13: models checking models When there is no ground truth, no answer key, no expert beside you, no test that fails red, you can still get an objective signal on quality by making models grade each other. Different models have different blind spots: trained on overlapping but not identical data, with different reward signals, by teams that emphasized different things. A point one model misses, a second often catches; the disagreement between them is the signal no single model gives. This only works across genuinely different **families**: Anthropic, OpenAI, Google, xAI, Meta, DeepSeek. Two Claude models cross-checking each other is not cross-model checking. [^af-03] Start with the light version. If you have one tool open, the single-model self-critique loop below gives most of the benefit. The full recipe is the high-stakes version, worth its setup only when being wrong is expensive: [^af-03] 1. **Start with the best model you have access to** for your kind of task, using the leaderboard as a starting point plus your own A/B test: the same prompt to two or three models, answers side by side. 2. **Generate the first draft with full context.** Brief it like a colleague, turn on thinking mode for hard problems, use the loop for structure. 3. **Ask it to grade its own output, 1 to 10, against named criteria.** Not "is this good?" but "score this on clarity, accuracy, structure, and what is missing, 1-10 each, with a one-sentence justification per score." The first grade is usually 7 or 8. 4. **Ask it to implement its own suggestions.** Repeat until the grade stops climbing, usually around 9. 5. **Take the draft to a second model from a different family. Ask for the same rubric.** It will catch things the first model graded itself on. 6. **Bring the second model's critique back to the first.** "Another model produced this critique. Evaluate which points are worth adopting, and why. Reject anything you disagree with, and explain." The first model adjudicates; you watch. 7. **For high-stakes work, repeat with a third model from a third family.** 8. **Stop when the score crosses your target across two independent models.** A 9.5 from your primary alone is not the same as a 9 from your primary plus a 9 from a different family. The second number is the one that means something. ### The single-model self-critique loop Steps 3 and 4 are usable on their own. A weekly status update, a slightly tricky email, a one-page memo: all get visibly better from one pass of "score this 1-10 against this rubric, then implement your own suggestions." A higher-leverage variant: **set a numerical target and let the model iterate autonomously toward it.** "Iterate against your own rubric until you reach 9.5 across all criteria, then show me the final version." The model grades, revises, regrades, five or six rounds in one response, and returns when it hits the target or plateaus. The target is a steering mechanism: 9 forces a different ceiling than 9.5, and 10 forces it to keep finding things until it genuinely cannot. [^af-03] This does not contradict the sycophancy concept. The difference is the rubric: without one, "is this good?" returns "great work"; with named criteria scored 1 to 10, the model has to point at what is missing from the other points, and that pointer is what you implement against. Forcing a number is not about the number; it is about what producing the number requires. "Pretty good" has to become "loses 1 point on structure because the third section repeats the second; loses 2 points on evidence because three claims have no source." Specificity is what you can act on, and the only readable signal to compare iteration N against N+1. [^af-03] **A privacy note.** Cross-model checking means pasting your draft into multiple tools. Check each tool's data policy before doing this with sensitive material; some do not train on your inputs, and some consumer products may by default. **An honest caveat:** three models can still all be wrong about the same thing, because they share more training data than you would guess. The score is a progress signal, not a truth signal. For anything legal, medical, financial, or about a real person, no number of cross-model passes replaces a human expert reviewing the load-bearing claims. Models check each other for craft; humans check the facts that matter. [^af-03] **When to skip the loop.** A short email, a quick lookup, a casual brainstorm: single-model is fine. The rule of thumb: if a thoughtful colleague would have spent two hours reviewing this, it earns the loop. [^af-03] The book's example: a consultant's 40-page strategy memo plateaued at 9 in her strongest model. A second family gave it 7.5 and listed eleven specific issues, three of which the first model had never raised. She fed those back to the first model to adjudicate; it adopted seven and rejected four with reasons. A third family surfaced two more. The counter-arguments she would never have seen on her own, because her primary model shared her blind spots, were in the memo before the board meeting. [^af-03] Continue to [twelve prompts to try](twelve-prompts.md). [^af-03]: AI Prompting 2026: 13 Concepts, 80% of Real Use, Part 4 "Working safely and choosing tools", Concepts 11, 12 and 13, including "The single-model self-critique loop, by itself". The AI Agent Factory, Panaversity, 2026. [^arena]: Arena leaderboard, as the book's recommended monthly check.