A Megantic interactive explainer
ChatGPT, Gemini and Google's AI Mode now sit between your customers and your product pages. In this walkthrough we follow a query through the whole machine, from raw text to grounded answer, so you can see exactly where an eCommerce brand earns its place in the response.
Start with tokenisationEvery example on this page is currently set to Home & Living. Change the niche in the top bar and the whole walkthrough adapts.
Step 1 of 10
Before a language model can do anything with a product query, it chops the text into tokens. A token is a frequently occurring chunk of characters, sometimes a whole word, sometimes a fragment of one. Common words like "shipping" usually survive intact, while rarer or compound words get split into smaller pieces the model has seen many times before.
This matters for eCommerce more than most people expect, because product vocabulary is full of exactly the kind of words that fragment: brand names, model numbers, materials and niche category terms. Every token then gets looked up as an ID in the model's vocabulary, and those IDs are all the model ever sees. Try it below with your own product queries.
The splits and IDs here are illustrative rather than a real tokeniser, but the behaviour is faithful: common retail words stay whole, unusual ones fragment, and punctuation, symbols and numbers become their own tokens. Notice how a price like "$200" costs two tokens on its own, the symbol and the number.
Step 2 of 10
Each token ID is then mapped to an embedding, a long list of numbers that acts like a coordinate. Real models use hundreds or thousands of dimensions; we can only draw three, but the idea carries over perfectly. Words that appear in similar contexts end up close together, so two synonyms sit near each other even when they share no letters.
This is the machinery behind semantic search and topical relevance. When a retrieval system compares your category page to a query, it compares positions in this space. Drag the space around, and tap any term to see its nearest neighbours and how similar the model considers them.
Drag to rotate, scroll or pinch to zoom. We placed the terms in clusters the way a trained embedding model would arrange them.
Step 3 of 10
Pretraining is a simple game played at enormous scale. The model reads a snippet of real text from the web, guesses the next token, gets told how wrong it was, and nudges its internal weights so the same mistake shrinks next time. Repeat this across trillions of tokens of web pages, product descriptions, reviews and forums, and the model gradually absorbs grammar, facts, brand associations and category knowledge.
Step through the example below, a line of copy from a typical product page in this niche, and flip between the untrained and trained model. Untrained, its guesses are close to random. Trained, probability piles up on the tokens that genuinely tend to follow, which is why well-written, widely referenced content about your brand literally shapes what these models expect to say next.
Pretraining teaches the model what text looks like; it does not teach it to be helpful. So a second phase follows, usually reinforcement learning from human feedback. Human reviewers compare pairs of model answers, prefer one, and the model is rewarded for producing answers people prefer: accurate, well structured, appropriately cited. This phase is why assistants lean towards recommending specific, well-supported entities rather than vague prose, and it quietly raises the bar for the sources they are willing to name.
Want to chat with a human to make sense of all this? Book in a chat with Evan!
Book a time with EvanStep 4 of 10
When someone asks an assistant a shopping question, the model rarely runs that question as a single search. It decomposes the request into a set of narrower sub-queries, runs them in parallel, and synthesises the results into one answer. Google calls this query fan-out in AI Mode, and researchers like Suganthan Mohanadasan have observed the same decomposition behaviour in ChatGPT's raw network traffic.
The practical consequence is generous: you no longer need to rank for one head term to be in the answer. You need content that genuinely resolves some of those narrower needs. Choose a query below and watch it fan out, then look at the mapping underneath, because each branch corresponds to a real customer need, and the pages that serve that need well are the ones that get cited. We come back to those candidate pages in Step 7, where retrieval is no longer enough: the model still has to decide which evidence survives into the final answer.
The sub-queries are illustrative, modelled on the decomposition patterns seen in AI Mode and ChatGPT search traffic: comparisons, attribute refinements, budget cuts, reviews and local variants. The mapping cards show the page best placed to win each branch.
Step 5 of 10
Here is the part that decides who benefits from all that fan-out. Models keep track of entities: distinct things like brands, products, materials and places, connected to each other by relationships. Research into ChatGPT's search behaviour shows it often writes specific brand names into its own sub-queries before it fetches a single page. If your brand is a well-established entity for a category, you can be named in the fan-out itself; if it is not, you are relying on luck.
Entities are built through reinforcement, in the everyday sense of the word: the same facts about your brand repeated consistently across your site, your schema markup, retailer listings, reviews, press and forums. Every consistent co-occurrence of your brand with your category strengthens that association in both training data and retrieval. Play with the signals below and watch the entity graph firm up.
Likelihood the model treats your brand as a known entity for this category
The percentage is illustrative, but the direction is real: entity strength grows with consistent, independent corroboration, and strong entities get named in fan-out queries and answers.
Step 6 of 10
A model can answer in two ways. It can rely on its parametric memory, the knowledge baked into its weights during training, which is fast but frozen at the training cutoff and prone to confident guessing. Or it can use retrieval-augmented generation: fetch fresh documents first, read them, and ground the answer in what it just read, usually with citations. Modern assistants blend both, and query fan-out is exactly how they gather the documents for the retrieval side.
Run both paths below on the same question and compare what happens. Then look at the table for what each path asks of your website.
Parametric memory, no retrieval
Fan-out, fetch, ground, cite
| Training data (memory) | Retrieval (RAG) | |
|---|---|---|
| Freshness | Frozen at the training cutoff, often months or years old | Live, fetched at answer time |
| How you influence it | Long-term entity building: consistent mentions across the crawlable web, staying open to training crawlers like CCBot and GPTBot | Retrievable, extractable pages: clean HTML, fast responses, schema, clear headings, self-contained passages that answer sub-queries |
| Where fan-out fits | Strong entities get written into the fan-out queries themselves | Fan-out decides which sub-queries your pages compete for |
| Failure mode for brands | The model has never heard of you, so it recommends competitors from memory | Your pages block AI crawlers, render only in JavaScript, or bury the answer, so someone else gets cited |
| Timescale to move | Slow: shifts with each new model training run | Fast: improvements can be picked up as soon as pages are recrawled |
Step 7 of 10
Retrieval gives the model a candidate set, not a finished answer. It still has to decide which sources contain evidence that is relevant, current, specific enough to support the claim and easy to extract without guessing. A page can therefore be found in Step 6 and still disappear here.
This is the part that makes AEO broader than simply "rank and hope". Use the before and after switch below to change only Your Store. The other sources stay exactly the same. Then run the selection and watch a retrieved page either fall out of the evidence set or survive all the way into the cited answer.
All five made retrieval. Click any source to inspect why it is strong or weak.
The strongest three sources above the cut-off are carried into the model's working context.
The model writes from the evidence that survived, not from every page it retrieved.
This is an illustrative teaching model, not a published ranking formula from OpenAI, Google, Anthropic or any other provider. Real systems use different retrieval, reranking and synthesis methods, and can weigh many more signals. The lesson is the important part: retrieval creates candidates; source selection determines which evidence is actually trusted and used.
Step 8 of 10
Everything above describes how the machine reads, retrieves and chooses evidence. Megantic's first two stages decide whether there is anything on your site strong enough to survive that process. Stage 1, Research, maps search demand across every layer of your store, from category pages down to filters and individual products, and sizes the total market you could reach. Stage 2, Implementation, rebuilds your hierarchy so each pocket of demand has its own clean, indexable, self-contained page.
Those two ideas line up with the LLM pipeline almost one-to-one. Every fan-out branch in Step 4 wants a page that answers it. Every retrieval in Step 6 wants a passage it can lift. Step 7 then asks whether that passage is strong enough to survive source selection and make the answer. Stage 1 finds those branches in the data before the model ever asks; Stage 2 builds the pages the model will be looking for, and the technical and content work underneath makes sure a crawler can fetch them and a model can quote them. Explore each part below.
The grey dashed nodes are demand your store already sells into but has no dedicated page for: filters buried behind parameters, categories that exist only as a menu link, product attributes customers search for by name. Run the research and watch them light up with their monthly search volume, along with the totals a research summary reports back.
Pick a search a customer might make. On the left is the journey through a typical store where filters live behind URL parameters. On the right is the same journey after Stage 2, where that filter has become a static, indexable page with its own URL, H1, copy and schema. Each extra click costs roughly 7 to 10% of shoppers, and a page that search engines and AI crawlers cannot index cannot be retrieved at all.
Filter exists, but only as a parameter
Filter is now a page of its own
Stage 2 covers the hierarchy, categories, filters and more than 35 technical elements: clean URLs, H1s, titles, redirects, internal linking, structured data, templates and speed. The journey above shows the piece that matters most for LLM discoverability: the page now exists as something a crawler can fetch and a model can quote.
Search engines and AI assistants never see your page the way a shopper does. They fetch raw HTML, follow links, read robots rules and parse structured data. Flip the switch to compare the two versions of the same filter page, then open any row to read why that element decides whether the page can be retrieved and cited at all.
There is no single "AI crawler". Different user agents do different jobs: some collect training data, some fetch pages live to answer a question, and some act on a shopper's behalf. Megantic's usual starting point is to allow all of them, because every block removes the brand from a specific part of the pipeline on this page. Plenty of stores have blocked these agents without realising, often through a platform default or a security rule. Use the switches to see what each block costs, and check the robots.txt it produces against your own.
User agent names and behaviour change often, and not every crawler honours robots.txt, so we verify the current list before making changes on a client site. Training and retrieval crawlers are also separated by more than robots: rate limits, rendering and IP ranges all play a part.
A category page with nothing but a product grid gives a model nothing to lift. On the left is that page. Tick the content elements on the right and watch it turn into a page that answers the fan-out branches from Step 4 in self-contained passages, with the entity named consistently throughout. Each element explains why it earns its place.
Search engines and language models both keep a knowledge graph: a web of entities (brands, products, places, categories) joined by typed relationships such as sells, located in or same as. Your brand becomes a node in that graph when the facts about it are stated in machine-readable form on your own site and then corroborated by independent sources. Retrieval finds candidate passages, but source selection decides which one the model quotes, and how well-established your node is drives that decision more than anything else.
Start by switching on your own Organization schema to declare the facts, then add off-site sources and watch the graph, the metrics and the citation likelihood respond.
Declared on your site
Corroborated off-site
Likelihood of being chosen as a cited source
Megantic's Stage 3 covers link acquisition, link reclamation (turning existing unlinked mentions into links) and partnerships. The percentage is illustrative; the ordering of what helps reflects current citation research, where brand authority is the strongest single predictor of being cited.
Here is everything on this page in one simulation. Pick a customer question, decide which layers of Megantic's work are in place, and run it. The pipeline shows what happens at each stage of the assistant's process, and the table underneath tracks every fan-out branch from being identified in research through to being cited in the answer. Switch a layer off to see exactly where a brand falls out of the answer, and why.
Step 9 of 10
The next step in the same pipeline is already arriving. Shopping agents inside ChatGPT, Gemini and others are starting to complete purchases on a customer's behalf: find the product, confirm price and stock, check the returns policy, and pay. Standards are forming around this, including OpenAI and Stripe's Agentic Commerce Protocol, Google's Universal Commerce Protocol and the WebMCP draft at the W3C, and the details will keep moving. What stays constant is that an agent needs the same things a crawler and a model need, plus a checkout it can operate.
Send an agent to buy the product from this niche on the before and after store. Every stage it passes depends on something built in Step 8, which is why the technical and content foundations are also the agentic foundations.
Protocol names and crawler behaviour in this area change quickly. The stages an agent must clear are stable: discover, verify, trust, transact. Megantic's Agentic Readiness Score in Step 10 audits exactly these.
Need a hand getting your site prepped for Agentic Commerce? Have a chat with Evan today!
Tell us a little about your store and Evan will come back to you, usually within one business day.
Goes straight to evan@megantic.com.au. No newsletter, no list.
Step 10 of 10
Everything on this page is diagnosed, tracked or built with a set of proprietary tools that sit alongside the three-stage strategy. Each tool covers a different part of the pipeline you have just walked through. Tap a tool to see which steps it maps to and what it produces.
The pipeline you just walked through is the environment your store now competes in. Text becomes tokens, tokens become positions in meaning-space, training turns the web's writing into expectations, fan-out multiplies every question, entities decide who gets recognised, retrieval finds candidate evidence, and source selection decides what survives into the answer and earns a citation.
This is the thinking behind Megantic's AEO work. If you would like to see where your own brand stands in this pipeline, we would love to show you.
