A few years ago, the question that kept business owners up at night was simple: “Where do we rank on Google?” Today there’s a quieter, stranger question that matters just as much. When a potential customer opens ChatGPT and types “who’s the best roofer near me” or “what’s a reliable HVAC company on Long Island,” what does the AI actually say? Does it mention you? Does it describe what you do correctly? Or does it confidently recommend three of your competitors and never bring you up at all?
Most business owners have no idea, because they’ve never checked. And unlike a Google ranking, which you can glance at in a few seconds, AI chatbot answers are harder to pin down. They vary by phrasing, by platform, by the day of the week, and by what the model happens to have indexed. That’s exactly why a structured AI visibility audit matters. You can’t fix what you can’t see, and right now your brand has a reputation living inside ChatGPT, Gemini, Claude, and Perplexity whether you’ve ever thought about it or not. This guide walks you through auditing that reputation yourself, step by step, in plain English.
Why Auditing AI Chatbot Visibility Matters Now
Search behavior is splitting into two lanes. Plenty of people still type keywords into Google. But a growing group asks an AI assistant a full question and trusts the answer it gives back, often without clicking a single link. When an AI names a handful of businesses in response to “best plumber near me,” those named businesses get a level of trust that’s hard to buy. The AI feels neutral. It feels like a recommendation from a knowledgeable friend rather than an ad.
This is the core of what people now call GEO (generative engine optimization) and AEO (answer engine optimization). The goal is no longer just ranking a blue link; it’s being the brand the AI cites, recommends, and describes accurately. The first step toward that goal isn’t writing new content or chasing backlinks. It’s measuring where you stand today. An audit gives you a baseline, surfaces factual errors that are actively costing you trust, and tells you which platforms already like you and which have never heard of you.
There’s also a defensive reason. AI models sometimes get things wrong. They may list outdated hours, describe services you stopped offering, attach you to the wrong city, or confuse you with a similarly named company. If a chatbot is telling prospects something false about your business, that’s a problem you want to catch early rather than discover after you’ve lost the work.
What an AI Visibility Audit Actually Measures
Before you start typing prompts, it helps to know what you’re looking for. A good audit tracks a handful of distinct things, and lumping them together is where most DIY attempts go sideways.
- Presence: Does the AI mention your brand at all for the queries that matter to your business?
- Accuracy: When it does mention you, does it get the facts right? Location, services, specialties, hours, contact details.
- Sentiment and framing: Is your brand described as a strong choice, a budget option, a niche specialist, or an afterthought tacked on at the end of a list?
- Citations and sources: Which websites is the AI pulling from when it talks about you or your category? Your own site, directories, review platforms, news mentions?
- Competitive position: Who shows up instead of you, or alongside you, and why do they seem to be winning the AI’s attention?
Keep these five buckets in mind as separate scores. A brand can be highly present but inaccurately described. Another can be perfectly accurate but almost never surfaced. The fix is different in each case, so measuring them separately is what makes the audit actionable instead of just interesting.
Step 1: Build Your Prompt List Like a Real Customer
The single biggest mistake people make is testing only one prompt: their own brand name. Of course ChatGPT knows your company when you spell it out. The more revealing test is the one where you never mention yourself at all and see whether the AI brings you up unprompted.
Write 15 to 30 prompts that mirror how real customers talk. Group them into a few types:
- Branded prompts: “Tell me about [Your Company].” “Is [Your Company] a good choice for [service]?” These test accuracy and framing.
- Category and local prompts: “Best [service] company in [your town].” “Who should I hire for [problem] near [area]?” These test presence and competitive position. This is where you find out if you exist in the AI’s world.
- Problem-first prompts: “My basement keeps flooding, who can help?” “I need a website for my small business, where do I start?” Customers often describe a problem, not a service category, so test that language too.
- Comparison prompts: “[Your Company] vs [Competitor].” “What are the top options for [service] on Long Island?” These reveal how you stack up directly.
Write these down in a spreadsheet before you run anything. You’ll reuse this exact list every time you re-audit, and consistency is what lets you see whether your visibility is improving over time.
Step 2: Run the Same Prompts Across Every Major Chatbot
Visibility is not uniform across platforms, and that surprises people. Each AI assistant draws on different data, different live-search tools, and different training. You might be a star in Perplexity and invisible in Gemini. So run your full prompt list through each one:
- ChatGPT (with web search enabled, since that changes results significantly)
- Google Gemini
- Perplexity (excellent for this because it shows its sources openly)
- Claude
- Microsoft Copilot, if your audience leans business or Windows-heavy
A few practical notes. Use a logged-out or fresh session where possible so your own history doesn’t bias the answers. Run each prompt once cleanly, then consider running it a second time later in the week, because answers drift. And pay special attention to the platforms that show citations, like Perplexity, because those sources are a roadmap to where your future GEO work needs to focus. If a competitor keeps getting cited from a particular directory or review site, that’s a place you need a strong, accurate presence too.
Understanding how AI assistants pull and cite sources is the foundation of all of this, and it’s the heart of what our AI SEO and GEO services are built around. If running five platforms by hand feels like a lot, that’s exactly the kind of ongoing monitoring worth systematizing.
Step 3: Score and Record What You Find
Now turn impressions into data. For each prompt on each platform, log a simple set of fields in your spreadsheet so patterns jump out:
- Mentioned? (Yes/No) for non-branded prompts especially.
- Position: Were you first, buried in a list, or a footnote?
- Accuracy notes: Anything factually wrong, outdated, or misattributed.
- Sentiment: Positive, neutral, or lukewarm framing.
- Competitors named: Who else showed up.
- Sources cited: Where the AI got its information, when visible.
Copy and paste the actual AI responses into a notes column. You’ll want the raw text later, both to track changes and to document errors. Once you’ve filled the grid, step back and look for the story. Maybe you’re accurate everywhere but only present for branded searches, which means people who already know you find good info but new prospects never discover you. Maybe one platform consistently lists a wrong service area. Maybe a single competitor dominates every category prompt. The grid turns a vague worry into a specific, fixable list.
Step 4: Diagnose Why the AI Says What It Says
Once you have results, the natural question is “why?” AI assistants assemble answers from a blend of their training data, live web search, and authoritative reference sources. When your brand is missing or misrepresented, it usually traces back to one of a few causes.
Thin or unclear information on your own site
If your website doesn’t clearly state who you are, what you do, where you serve, and what makes you distinct in plain, factual language, the AI has nothing solid to anchor to. Vague marketing copy that’s heavy on adjectives and light on facts is hard for a model to summarize accurately. Clear service pages, a strong about page, and explicit location and contact details give the AI clean material to work with.
Weak or inconsistent presence across the web
AI models cross-reference. If your business name, address, and details are inconsistent across directories, your Google Business Profile, review sites, and your own pages, the model gets conflicting signals and may default to whoever looks most consistent and authoritative, which is often a competitor. Consistency across the wider web is as important for AI as it ever was for traditional local SEO.
Outdated or missing structured information
Models love clear, structured, well-organized facts. Pages that answer real questions directly, FAQ content, and properly marked-up business information all make it easier for an AI to extract correct details. If your most important facts only live inside an image, a PDF, or a slick interactive widget, the AI may simply not see them.
Sorting out which of these is your bottleneck is genuinely strategic work, and it’s where bringing in outside help pays off. Our AI consulting services exist to turn an audit’s raw findings into a prioritized plan rather than a pile of observations.
Step 5: Turn Findings Into a Fix List
An audit that ends in a spreadsheet is wasted effort. The point is action. Sort what you found into three priority tiers.
- Urgent (factual errors): Anything the AI states that is flat-out wrong, especially about location, services, safety, or contact details. These directly mislead customers and undermine trust. Fix the underlying sources first, your own site, your Google Business Profile, major directories, so the correct information is the dominant signal.
- High-value (presence gaps): Category and local prompts where you don’t appear at all. This is where the real growth lives. Closing these gaps usually means publishing genuinely helpful, specific content that answers the exact questions customers ask, and earning mentions on the sources AI tends to trust.
- Polish (framing and depth): Cases where you appear but get a thin or lukewarm description. Strengthen the story your web presence tells about your specialties, your service area, and what makes you the right call.
Be patient with the timeline. AI models don’t update the instant you change a page the way a Google crawl might. Some platforms refresh quickly through live search; others lag because they lean on training data. Make the changes, then re-run your audit in 30 to 60 days and compare against your baseline grid. Improvement here is a flywheel, not a switch.
How Often Should You Re-Audit?
For most small and mid-sized businesses, a full audit once a quarter is a sensible rhythm, with a lighter monthly spot-check on your handful of most important prompts. If you’re in a fast-moving or competitive category, or you’ve just made significant changes to your site or services, check more often. The key is consistency: same prompts, same platforms, same scoring, so you’re comparing like with like and can actually prove movement.
This is also where the “empower people, don’t replace them” mindset comes in. You can absolutely run this audit yourself, and you should at least once, because nothing teaches you more about how AI sees your business than watching it answer questions about you in real time. The goal of bringing in tools or a partner later isn’t to take the work off your plate so you stop understanding it. It’s to handle the repetitive monitoring and technical fixes so your team can focus on the parts only humans do well: judgment, relationships, and delivering work worth recommending.
From Audit to Action
Auditing your AI visibility isn’t a one-time stunt; it’s becoming a normal part of running a business that wants to be found. The companies that win the next few years aren’t necessarily the ones with the slickest sites. They’re the ones AI assistants describe accurately, recommend confidently, and cite by name when a real person asks for help. You can’t influence that until you’ve measured it, and now you have a repeatable process to do exactly that.
If you’d rather not piece this together alone, or you’ve run the audit and don’t love what you found, we can help you fix it and keep it fixed. Take a look at our AI business integration services to see how MJW Media helps Long Island and beyond turn AI visibility from a blind spot into a steady source of new customers, and reach out for a conversation about where your brand stands today.
What is an AI visibility audit?
An AI visibility audit is a structured check of how AI chatbots like ChatGPT, Gemini, Perplexity, and Claude describe and recommend your business. It measures whether you’re mentioned for relevant questions, whether the facts are accurate, how you’re framed against competitors, and which sources the AI pulls from. The result is a baseline you can act on and re-measure over time.
Which AI chatbots should I test my brand on?
At minimum, test ChatGPT (with web search on), Google Gemini, Perplexity, and Claude. Add Microsoft Copilot if your audience skews business or Windows-heavy. Perplexity is especially useful because it openly shows its sources, which tells you exactly where your information is coming from and where to focus your improvements.
Why does ChatGPT recommend my competitors but not me?
Usually it comes down to clarity and consistency of information. If your website states your services, location, and specialties in plain factual language, and that matches what directories, your Google Business Profile, and review sites say, the AI has strong signals to work with. Competitors often win simply because their information is clearer, more consistent, and appears on sources the AI trusts.
How long does it take to improve AI visibility after making changes?
It varies by platform. Tools that rely heavily on live web search can reflect updates within days, while models that lean on training data may take weeks or longer. A practical approach is to make your fixes, then re-run the same audit in 30 to 60 days and compare against your baseline to confirm real movement.
Can I do an AI visibility audit myself or do I need help?
You can absolutely run a basic audit yourself, and doing it at least once is valuable because you learn firsthand how AI describes your business. The work becomes harder to sustain when you’re tracking many prompts across several platforms every quarter and diagnosing technical causes. That’s when a partner like MJW Media can take over the monitoring and fixes while you stay informed.
