The AI search KPIs that survive the first board meeting

A new metric gets into a report easily and leaves just as easily. What survives is what changes a decision.

The rush to measure AI visibility produced a crop of metrics nobody knows how to use, most of them gorgeous in a screenshot and silent in a meeting. The question that separates the ones that stay is blunt: when this number changes, what does the company do differently.

I organize measurement by the five layers of the Cognitive Architecture of SEO, with eighteen metrics that answer that question. The layers are cumulative: a failure in the first one cancels the others, and reading them out of order is the mistake behind most wrong conclusions. Layer 1 answers whether the system reaches the content. Layer 2, whether the content can be retrieved in parts. Layer 3, whether the system understands what it is about and whom it belongs to. Layer 4, whether the brand is remembered when the answer generates no click. Layer 5, whether the person or the agent decides based on it.

Layer 1. Crawlability, the access KPIs

Before measuring appearance, you need to know whether the content can be reached. This layer accounts for absence with a technical cause, and it is the only one that separates a brand that is invisible because it is blocked from a brand that is invisible for lack of authority. What each AI crawler does, and what you lose by blocking each one, is in the AI bots guide.

• Share of templates accessible to AI crawlers. Definition: among the templates that carry the content of the territory, the share that returns complete content to the agent of each target system, checked template by template. Decision it supports: open or keep a block, and prioritize infrastructure fixes.
• Coverage of declared blocks. Definition: list of AI agents with an explicit directive in robots.txt and at the edge layer, with the date of the last review. Decision: align the access policy across teams, which tends to be inconsistent between SEO, infrastructure and legal.

Layer 2. Structure, the legibility KPIs

With the crawler inside the page, the question becomes whether it can tell price from author from specification. This is the best documented of the five layers, because schema, authorship markup, rendering and page architecture have plenty of official documentation. Two metrics sum up its state: one for what is marked up, another for what reaches the crawler without depending on execution.

• Coverage of valid structured data on critical templates. Definition: share of the territory’s templates that publish valid schema, with the Organization or Person referenced by @id from the other blocks. Decision: the order in which templates get fixed and what goes into the next version of the site.
• Share of critical content that depends on JavaScript execution. Definition: share of the text that carries the value proposition and is not in the source HTML. Decision: move content to server-side rendering, and in which template order.

Layer 3. Interpretation, the framing KPIs

The question here is whether the system understands what the brand is about and which category it belongs to. Showing up with the wrong description is worse than not showing up, because the description circulates and feeds the next answers. This is the layer that demands the most human reading, and the one almost no dashboard delivers ready-made.

From here on, collection depends on sampling: a frozen prompt set and repeated runs per prompt, because the answer varies between executions.

• Distance between the page and the shape of the question. Definition: cosine similarity between the page text and the questions in the set, calculated with the same model every cycle and read as direction, with no universal threshold. The cosine similarity calculator runs the math for a pair of texts. Decision: which pages to rewrite in the vocabulary of the people asking.
• Attributes associated with the name. Definition: the adjectives and categories that come along with the brand, logged verbatim and grouped later, without summarizing at collection time. Decision: which wordings to fix at the source, and which ones to reinforce.
• Competitive set the brand is placed in. Definition: the names cited alongside it, with the frequency of each. Decision: confirm or fix the category positioning, because being listed among competitors from another tier signals category confusion.

Layer 4. Memory, the presence KPIs

What gets measured here is whether the brand is remembered as the same entity, across platforms and over time. A single appearance records an event, and stable appearance across cycles is what indicates memory.

• Entity consistency across official sources. Definition: share of official profiles and sources that use the canonical spelling of the name and point to the same entity, checked against a closed list of sources, with a date. Decision: which profiles to fix first and which variations to declare in alternateName.
• Appearance rate in the prompt set. Definition: share of runs in which the brand is cited, over the total runs in the period, with the prompt set and the number of repetitions declared. Broken down by platform, the same number shows in which systems the brand is missing. Decision: keep or redirect content investment in the territory.
• Relative share against competitors in the territory. Definition: the brand’s appearance rate divided by the sum of the rates of the real competitive set, defined by the brands that compete for the same questions. Decision: reading of competitive position and choice of whom to monitor.
• Recurrence of appearance across cycles. Definition: share of prompts in which the brand appears in two consecutive collection cycles, over the prompts in which it appeared in the first. Decision: separate stable presence from occasional appearance before reporting a gain.
• Share of mentions backed by your own source versus a third-party source. Definition: among answers that cite a source, the split between your own domain and third-party domains. Decision: balance investment between owned content and external presence, since relying only on your own source is fragile.

Layer 5. Action, the choice and consequence KPIs

The last layer answers whether the person or the agent decides based on the answer, and the typical failure mode is the brand that gets cited and never becomes the final recommendation. It combines metrics measured inside the answer and metrics measured after it.

This is the layer the board asks about first, and the one with the most imperfect measurement of the five. It has to be presented with its limitation declared, or the first pushback will take down the whole report.

• Position inside the answer. Definition: classification in three bands, first recommendation, middle of the list and passing mention, logged per run. Decision: whether the job is getting onto the list or moving up within it, which call for different actions.
• Presence in decision questions compared with top-of-funnel questions. Definition: the same rate, calculated in two cuts, with comparison and objection questions on one side and definition questions on the other. Decision: where to reinforce content, since showing up only at the top indicates recognition without a way into the choice.
• Coverage of trust signals verifiable by a third party. Definition: share of the page’s authority claims, such as authorship, credentials, reviews and awards, that have confirmation in an external source that can be checked. Decision: where to invest in external presence, including reviews on third-party platforms, since reviews a brand publishes about itself are not eligible for review stars.
• Referral traffic from AI platforms. Definition: sessions whose identified origin belongs to an AI platform, with the list of origins used declared in the report. Decision: size the channel and justify continued investment.
• Behavior of people who arrive this way. Definition: pages per session, depth and goal completion rate, compared with the same set coming from organic search in the same period. Decision: adjust the landing page, because a visitor who arrives already informed needs something else on screen.
• Presence of the channel in logged sales conversations. Definition: share of opportunities in which the contact says they found or checked the brand in an assistant, logged in a dedicated CRM field filled in during the conversation. Decision: sustain or review the channel’s budget. It is the most important metric across the five layers, because it is the data the board accepts as proof.

The imperfection of this layer has a name. Part of the traffic coming from assistants arrives without an identified origin, which systematically underestimates the channel. The practical consequence is that the number works for tracking direction over time with the same collection method, and is unfit for claiming the absolute size of the channel. Declaring this before someone asks costs less than explaining it afterwards.

Report cadence and format

The five layers do not need the same frequency. Layers 1 and 2 are checked at every relevant release of the site and reviewed in full once a month, along with entity consistency and trust signals. Layers 3 and 4 and the choice metrics in layer 5 are collected in the same round, because they come from the same sample, every two weeks outside periods of change. The consequence metrics close monthly, together with the sales report.

The format that survives the meeting has one page per layer, with the metric, the value, the change against the previous period, the pending decision and the owner of that decision. A metric without a declared owner loses its practical effect, because the decision it supports has nowhere to go.

The eighteen metrics live in a spreadsheet with the collection log and the calculations ready: four of them come out on their own from the logged runs. You can make a copy in Google Sheets or download it as Excel. The spreadsheet labels are in Portuguese for now, and the formulas speak every language.

The AI KPIs I would cut from the report

• Raw mention count without a prompt breakdown. It goes up and down without telling you anything.
• Aggregate sentiment without attribute reading. It hides the most expensive problem, which is being described correctly, in a positive tone, inside the wrong category.
• Comparison with a competitor that does not fight for the same territory, chosen because it is famous in the market.
• A composite visibility score presented on its own. It is useful for tracking direction, and used as the single goal it encourages optimizing the index instead of what it summarizes.
• A one-off reading without repetition. One run per prompt records one possible occurrence, without supporting any claim about change between periods.
• Any metric whose operational definition does not fit in one sentence. A definition that needs a paragraph means the metric is not ready to enter the report.