More and more of our conversations with Long Island business owners start the same way: "So I asked Claude about this, and it said…" Sometimes it’s ChatGPT or Gemini. The advice that follows varies. Our homepage needs more words. We should be posting on Google every day. Our old blog posts are hurting us and should be deleted. We need a new website.
Sometimes that advice is right. Sometimes it’s exactly what we suggested months ago. And sometimes it’s confidently wrong for that particular business. We’re not here to tell anyone to stop using AI. We use it every day, and we teach teams how to use it well. But there’s one thing worth understanding before you act on what an AI tool tells you: it answered the question you typed, not the business you run.
We’ve written before about how to catch AI hallucinations, the made-up facts and fake sources AI tools sometimes produce. This is a different problem, and a more common one. Most of the time the AI isn’t making things up. It’s giving you a perfectly reasonable answer to a question it doesn’t have enough information to answer.
The AI Only Knows What You Told It
Picture asking a smart stranger at a networking event, "Why are fewer people calling my business?" They’ve never seen your website, your Google listing, or your numbers. They don’t know you moved locations in the spring or that a new competitor opened two towns over. They’d give you general advice: improve your reviews, update your website, post more on social media. Not bad advice. Just not your advice.
That’s the position an AI assistant is in every time you ask it about your business. Here’s what it typically can’t see:
- Your actual data. Google Search Console, Google Analytics, your call tracking and your Google Business Profile insights show what really changed, when, and where. The AI is guessing at all of it.
- What’s happening in your market. A Google update, a seasonal dip, a competitor’s new ad campaign, or a change in how Google shows local results for your type of business.
- What’s already been tried. If a change was tested last year and didn’t help, the AI has no way of knowing that.
- Why things are the way they are. That odd page might exist because your industry requires certain disclosures, or because removing it would break links that bring in customers.
- Your goals. More calls, more online orders, bigger jobs, fewer tire-kickers. The right answer changes depending on which one you care about.
Some AI tools can search the web, and some will look at your website if you give them the link. That helps. But a quick look at your public site still isn’t the same as seeing the numbers behind it.
Why the Answer Sounds So Certain
If the AI is working with so little, why does it sound like an expert who has studied your business for a month?
Because AI tools are built to give clear, organized, helpful answers, and they don’t naturally signal when they’re unsure. A person who isn’t sure will say "it depends" or ask you a few questions first. An AI assistant usually gives you a neat list of five recommendations in the same calm tone it uses for things it knows for certain. The answer sounds fluent and polished, and that’s easy to mistake for being correct.
There’s a second habit that matters even more. AI tools tend to agree with the way you ask the question. Ask "Is my website why sales are down?" and it will usually find reasons your website might be the problem. Ask "What else besides my website could explain lower sales?" and it will happily give you a different list. Researchers call this tendency sycophancy, and it’s a known trait of these tools. Your question shapes the answer before the AI knows a single fact about you.
Here’s a simple test you can run today. Ask the same question twice, once leaning one way and once leaning the other. If the answer flips, you’ve learned how much of it came from your wording.
Why It’s So Easy to Believe
Notice something next time you use one of these tools. Ask it about your own line of work, whether that’s plumbing, dentistry, law or landscaping, and you’ll probably spot the oversimplifications right away. "That’s not how it works in real life," you’ll think.
Then ask it about SEO, website code or online advertising, and it sounds like it knows exactly what it’s talking about.
That’s not a flaw in you. You run a business; you’re not supposed to be an expert in search engines. But it means AI advice is easiest to believe in exactly the areas where you’re least able to check it. Outside your own expertise, a wrong answer and a right answer sound identical.
General Advice vs. Advice for Your Business
Much of what AI tools recommend is "best practice": the advice that shows up most often across the articles and forums they learned from. Best practice is a decent starting point. It’s also, by definition, the average answer for the average business.
Your business isn’t average. A personal injury firm in Nassau County, a family restaurant in Suffolk and a contractor serving the whole Island might ask an AI the same question and should get three different answers. "Write longer pages" might help one, do nothing for another, and hurt the third if it pushes the phone number further down the page on a phone.
Good SEO and marketing work happens in the space between "what usually works" and "what works for you." That space is filled with your data, your history and your goals, which is exactly what the AI didn’t have.
How to Get Better Answers From AI
You can make AI tools far more useful with a few habits:
- Give it real context. Tell it your industry, your town, who your customers are, what’s changed recently and what you’ve already tried. The more it knows, the less it has to guess.
- Ask what it’s missing. Follow up with: "What would you need to know to be confident in this advice?" Whatever it lists is what it was guessing about.
- Ask for the other side. "What’s the strongest argument against doing this?" Advice that falls apart under that question wasn’t solid to begin with.
- Keep the question neutral. "What could cause fewer calls from my website?" gets a more balanced answer than "Is my website losing me calls?"
- Treat the answer as a question, not a verdict. The most useful thing an AI answer gives you is a sharper question to bring to someone who can see your actual numbers.
If your team is using these tools every day, it’s worth learning these habits together. That’s a big part of what we cover in our AI training for teams.
Bring Us What Claude Said
If you work with us and an AI tool tells you something about your website or marketing, please send it over. We’d much rather talk it through than have you quietly wonder whether we missed something. A few things make that conversation fast and useful:
- Send the whole exchange, including exactly what you asked. The wording often explains the answer, and the caveats tend to disappear in a summary.
- Tell us what you gave it. A link, a screenshot, a report, or a description from memory all lead to different answers.
- Ask us what your data says. We can usually check an AI’s claim against your real numbers quickly. If the data backs it up, we’ll act on it. If it doesn’t, we’ll show you why.
And sometimes the AI will be right. When it catches something real, we’ll say so. The point isn’t to win an argument with a chatbot; it’s to get to the right answer for your business.
The Bottom Line
AI assistants are genuinely useful, and they’re improving fast. But an AI answer is only ever as good as the information behind it, and when you ask about your business, that information is usually a sentence or two. The tool won’t tell you what it’s missing. It will just sound sure.
So keep asking it questions. Then put the answers next to the full picture before you make a decision. If you’d like a second set of eyes on something an AI told you, or help getting your team to use these tools with confidence, get in touch with MJW Media.
Frequently Asked Questions
Can I trust AI advice about my website or SEO?
AI advice is often reasonable as a general starting point, but it’s based on common patterns across the internet, not on your data. Without your analytics, history and goals, an AI tool can’t know whether general advice fits your business. Treat it as something to discuss and verify, not a final answer.
Why does AI sound so confident even when it’s wrong?
AI tools are designed to give clear, organized, helpful answers, and they don’t naturally signal uncertainty the way a person would. A guess and a well-supported fact often come out in the same confident tone, which makes them hard to tell apart if you don’t already know the subject.
Does the way I word a question change what AI tells me?
Yes. AI tools tend to agree with the framing of a question, a tendency known as sycophancy. If your question assumes something is a problem, the answer will usually confirm it. Neutral, open-ended questions get more balanced answers.
What’s the difference between this and an AI hallucination?
A hallucination is when an AI invents something false, like a fake statistic or a source that doesn’t exist. Missing context is different: the AI’s advice may be accurate in general but wrong for your business, because it doesn’t know your data, your market or what you’ve already tried.
Should I share what an AI tool told me with my marketing agency?
Yes. Send the full exchange, including your exact question, so your agency can check the advice against your real data. A good agency will tell you when the AI got it right and explain clearly when it was missing context.
