Your mortgage calculator and your tip splitter are not equally exposed to ChatGPT, and treating them the same wastes effort. OpenAI's announcement says users can ask ChatGPT to build a tool inside the conversation. It names a savings-growth calculator and a dinner bill splitter as examples (Search Engine Journal). The useful question is which of your pages look like those examples and which look nothing like them.
This guide gives you a repeatable audit. You score each tool page on five criteria, check the score against your own Search Console data and a hands-on prompt test, then assign one action: keep, differentiate, or consolidate.
Step 1: List every tool page and its target queries
Export every calculator, converter, estimator, and generator URL on the site. Next to each, write the two or three queries it should win, such as "savings calculator" or "cups to grams". You will reuse these queries in both verification steps.
Group pages by tool family. Ten near-identical unit converters count as one family for the consolidation decision later.
Step 2: Score each page on five criteria
Score each criterion from 0 (low replacement risk) to 2 (high replacement risk). A higher total means the page is easier to recreate in a chat.
| Criterion | 0 (defensible) | 1 (mixed) | 2 (easily replicated) | |---|---|---|---| | Input complexity | Many dependent fields, uploads, or branching scenarios | Several fields that interact | Four or fewer generic inputs | | Proprietary data or rates | Your own data or live local rates drive the result | Public data that needs periodic lookup | Public formula only | | Need for trust or accuracy | Financial, medical, or legal result a user may act on | Money- or health-adjacent, approximate output | Low stakes, a wrong answer costs little | | Saved or shared output | Accounts, saved scenarios, shareable links, exports | Printable or copyable result | Result used once and discarded | | Integrations | Connects to accounts, inventory, booking, or lender systems | Light pre-fill or embed | Standalone, no connections |
Why these five criteria
Search Engine Journal's report points at the pattern. It describes OpenAI's savings calculator and bill splitter as tools that use generic inputs and public math. The author adds that in August this type of tool looked easiest for Google's generative UI to replicate in AI Overviews. That is the author's observation, not a measurement.
The five criteria extend it. A page sits further from that pattern if it needs data a chat model cannot reliably hold, produces a result the user must trust, or gives the user a place to store and send the output.
Step 3: Apply the decision rule
Add the five scores. The thresholds below are a working rule of thumb for triage, not an industry standard.
- 0 to 3: keep. The page has real moats. Maintain it, keep the data fresh, and make the moats visible on the page (data source, update date, save or share function).
- 4 to 6: differentiate. Part of the page is replicable. Add whatever scored 0 or 1 elsewhere: proprietary data, scenario saving, expert review, or an integration.
- 7 to 10: consolidate. A chat can likely reproduce the tool. Fold thin tool pages into a stronger guide or hub where the calculator supports explanation, examples, and decisions. Merge near-duplicates into one URL with redirects.
One adjustment: if Step 4 shows a clear decline and Step 5 shows ChatGPT reproducing the tool, move the page up one tier of urgency. If neither check shows a problem, hold the page where its score puts it and recheck later. For more on this, see more on how to build an ai visibility prompt set by tag (template).
Worked example: savings calculator vs. mortgage tool
This example is illustrative. The pages are hypothetical, the scores are judgment calls, and no traffic or ranking data is behind them.
Generic savings calculator. The inputs are starting balance, monthly deposit, interest rate, and years (2). The math is a public compound-interest formula (2). The output is a projection, money-adjacent but approximate (1). Nothing is saved or shared (2). It connects to nothing (2).
Total: 9. The rule says consolidate. The page mirrors the savings-growth example OpenAI itself used, so the standalone tool is the part most likely to be duplicated in chat. A sensible move is a guide on how much to save for specific goals, with the calculator embedded as one element.
Mortgage tool with live local rates. The inputs include price, down payment, term, taxes, insurance, credit tier, and ZIP code (1). Rates come from a live local feed (0). Users may act on a financial result (0). Users can save a scenario and send a link to a partner or agent (1). A lender quote request connects to outside systems (1).
Total: 3. The rule says keep. The work is protecting the moat: show the rate source and timestamp, and keep the feed reliable.
The gap between 9 and 3 comes mostly from data and trust, not from how the tool looks. A chat can draw a prettier interface than most small sites ship. Whether it can vouch for this week's rates in a specific ZIP code is a separate question, and your Step 5 test will show you.
Step 4: Check your Search Console trends
Scores are a hypothesis. Test them against your own data before moving any page. This pairs well with go deeper on how to fact-check ai-generated metadata before publishing.
- Open Performance, then Search results. Set the date range to the last three months compared with the previous period. For seasonal tools, compare year over year instead.
- Filter by page, using a URL pattern that matches your tool directory. Note clicks, impressions, CTR, and average position per page.
- Switch to the Queries tab with the same filter. Look at the target queries from Step 1, and at question-style variants such as "how much will I have if...".
- Read the pattern. Falling clicks with steady impressions suggest searchers see you but get what they need elsewhere. Falling impressions suggest demand or visibility has shifted. Flat numbers on both mean no evidence of loss yet.
- Annotate the date. The rollout began October 7, 2026, so earlier data cannot reflect it. A decline that started before then has another cause, and you should rule that out before blaming chat tools.
Search Console covers Google Search only. Check your analytics separately for referrals from chat products, and treat their absence as unproven rather than reassuring.
Also read: how to audit google ads cpc increases: a worked example
Step 5: Run a manual ChatGPT prompt test
Test every page scoring 4 to 10, plus any keep-page with falling clicks. Ask ChatGPT to build the same tool. Start with the exact query from Search Console, then try a direct prompt such as "Create an interactive calculator for [your tool] with these inputs."
Record these items for every test:
- Plan, model name, thinking level, and whether you used the web. Search Engine Journal reports that the rollout is staged by plan. It also reports that the Pro thinking level and older macOS and Windows desktop apps do not support Intelligent UI, so results will differ by account.
- Whether ChatGPT built an interactive tool or answered in text.
- Whether the result matches your tool's output on three test inputs, including one edge case.
- Whether it asked for or looked up current data, and where the numbers came from.
- Whether a Sources button or citations appeared, and whether they link to you or to anyone.
A tool that matches yours on all three inputs without extra data confirms a high score. A tool that guesses rates, skips edge cases, or cannot show a source supports a lower one. Repeat the test monthly, since behavior may change.
What nobody can tell you yet
OpenAI's GPT-6 announcement does not say how sources will appear when answers are presented as charts, comparisons, or tools. ChatGPT's search help page says answers that use web results may include citations through a Sources button when available. That is not a promise for generated tools. Whether your site gets credit, a link, or nothing when a chat builds a tool in your category is unknown.
Two other limits apply. Availability depends on plan and surface, so your audience's exposure is uneven. And no one can predict traffic impact from this news alone. Your scores and tests estimate exposure, not outcomes.
What to do next
Trust your own repeat tests over industry commentary. Re-score pages when your data or the tools change, and rerun the Search Console comparison after a full month of post-rollout data. If OpenAI documents citation behavior for generated tools, add a sixth check to the prompt test: whether the tool credits the page it drew on.
