Judging AI Claims Without the Hype

Lesson 10 · taught by Pip

You don't need to understand the engine to judge a car. You ask where it's been, how it handles, and what breaks first. AI is the same. With a handful of plain questions, you can tell a real offer from a shiny one — no jargon required.

Think of it like hiring. You wouldn't take someone on because their résumé used impressive words. You'd ask what they've actually done, where they get stuck, and who's responsible when something goes wrong. That same calm curiosity is your whole toolkit here.

The questions that cut through

"Where does it get its answers?" A good AI for your business answers from your knowledge — your services, your hours, your policies. If a pitch can't tell you what the tool actually draws on, it's guessing, and so are you.

"What happens when it doesn't know?" You've learned that AI can make things up (lesson 7). So ask plainly: does it say "I'm not sure, let me connect you to a person," or does it bluff? An honest tool that admits limits beats a confident one that invents.

"What's your accuracy number — and how did you measure it?" Beware round, heroic figures like "99% accurate." Accurate at what, on whose data, measured how? A real answer is specific and a little humble. A vague one is marketing.

"What can't it do?" This is the most revealing question of all. Anyone who says "it does everything" has told you they either don't understand the tool or hope you don't. The honest seller names the edges gladly — because knowing the edges is how you use it well.

Reading between the lines

Watch your language flags. Words like magic, fully autonomous, and no human needed are usually covering thin spots. The real thing tends to sound modest: "It handles the common questions reliably, flags the tricky ones to you, and gets better as you feed it more."

And ask the unglamorous questions too. Who owns your data? Can you see what it told customers? What does it cost when you grow — not just today? You covered safety in lesson 9; let that instinct ride along here. A trustworthy offer welcomes these questions. A hyped one hurries past them. 🔦

Your turn

Next time an AI tool is pitched to you, ask just two things: "Where does it get its answers?" and "What can't it do?" The quality and honesty of those two answers will tell you most of what you need to know.

That's the voyage. Across ten lessons you've learned enough to listen to any AI pitch with clear eyes — to know what's plausible, what's puffery, and what's worth your money. For where to sail next, the lighthouse keeps more courses lit at /lighthouse/courses. Fair winds. 🐙