How to Evaluate Amazon Agency Performance: The KPIs That Actually Matter

Why "Sales Are Up" Isn't a KPI

Every quarterly review starts the same way: a slide with a line going up and to the right. Ad sales climbed, campaigns launched, impressions grew. It feels like progress, and often it is, but "sales are up" on its own answers almost nothing about whether the agency running your account is actually doing good work or just spending more of your money to generate more top-line activity.

The distinction matters because both explanations produce the exact same slide. An agency that found genuinely efficient new keywords and an agency that simply raised bids across the board can both report higher ad sales this month. One of those is durable progress. The other is a temporary sugar high that quietly compresses your margin while looking, on the surface, identical to real growth. Telling the two apart requires a framework, not a bigger dashboard.

Part of the problem is that activity is genuinely easier to talk about than results. A rep can walk you through twelve new campaigns launched this month in five minutes. Explaining whether your actual contribution margin per unit improved takes longer, requires pulling numbers from more than one place, and doesn't always have a clean, satisfying answer. It's not that agencies are hiding the harder numbers on purpose, in a lot of cases nobody on either side of the table has agreed in advance on which numbers the relationship is actually being judged by. That ambiguity is exactly what a framework closes.

This article builds that framework: a three-tier hierarchy for sorting what you're shown into what actually matters, the specific metrics that belong in each tier, and a quarterly scorecard you can fill in yourself, independent of whatever your agency chooses to present. For the pre-signing version of this same question, our guide to what to ask an Amazon agency before you sign covers exactly what to ask about reporting before you're the one stuck decoding a vague update after the fact.

The Metric Hierarchy: Vanity Metrics, Profit Metrics, and Leading Indicators

Not every number an agency can show you carries the same weight, and treating them as interchangeable is how a genuinely busy quarter gets mistaken for a genuinely good one. It helps to sort everything into three tiers.

Vanity metrics sit at the base. Total ad sales, number of campaigns launched, impressions, clicks. These are easy to move, in some cases simply by spending more, and easy to report, which is exactly why they show up first in a lot of updates. They're not meaningless, a total collapse in impressions would be a real problem, but on their own they prove activity, not results.

Profit metrics sit in the middle. ACoS, and more importantly TACoS, along with anything tied directly to what's actually left over after ad spend. These are harder to inflate without consequence, since a metric built around efficiency punishes wasteful spending rather than rewarding it.

Leading indicators sit at the top, and they're the ones a narrow, PPC-only view misses most often. Organic rank movement on your priority keywords, organic sales share, and conversion rate trend all represent durable, compounding progress, the kind that keeps paying off even in a month where ad spend gets paused entirely.

None of the three tiers works in isolation, and none should be read without the other two for context. A quarter with strong vanity metrics, healthy profit metrics, and stalled leading indicators can still look like a success on paper, right up until the ad spend that's carrying the account gets reduced and there's no organic foundation underneath to catch the fall. The hierarchy isn't about ignoring the lower tiers, it's about refusing to let them stand in for the tiers above them.

ACoS Alone Isn't Enough: Why TACoS Belongs in Every Review

What each metric actually measures

ACoS, advertising cost of sales, measures ad spend against the sales your ads directly generated. It's useful for judging a specific campaign's efficiency in isolation. TACoS, total advertising cost of sales, measures total ad spend against your total sales, organic and paid combined. That distinction matters enormously, because ACoS can look perfectly healthy in a month where your organic sales quietly declined and ads simply picked up the slack, total revenue holding flat while the underlying business got weaker, not stronger.

Illustrative example: imagine a hypothetical brand where ad sales rise 20 percent quarter over quarter, a number that would look great in isolation. But TACoS rises right alongside it, meaning the growth came from spending proportionally more to get there, not from becoming more efficient. Top-line ad sales growth alone doesn't prove the business is more profitable, and in this case it isn't, it's just spending more to stand still.

Why an ACoS-only report is a signal worth noticing

An agency that reports ACoS every month and never mentions TACoS isn't necessarily doing anything wrong. But per industry commentary, this pattern is worth treating as a signal about what's actually being optimized, ad efficiency in isolation rather than whole-business profit. This is exactly what to ask about before you sign anywhere, covered directly in our questions-to-ask guide, and it's exactly the vanity-metrics pattern flagged in our breakdown of Amazon agency red flags as well. If you're not seeing TACoS in your current reporting, that's a reasonable, low-friction thing to ask for directly.

Our Amazon advertising team builds TACoS into every account review by default, precisely because ACoS in isolation has this blind spot built in.

Organic Growth Metrics an Agency Should Be Moving

PPC metrics get most of the attention because they update fastest and feel the most controllable. Organic metrics move slower, but they represent the part of your business that keeps generating sales even in a month you didn't spend a dollar on ads. An agency that never brings up organic performance unprompted is implicitly telling you where their attention is actually going, regardless of what the contract's scope of work says on paper.

Organic sales share

What percentage of total sales came from organic placement rather than sponsored placement, tracked over time. A healthy trend moves this share upward or holds it steady while total revenue grows. A shrinking organic share, even alongside rising total sales, usually means the business is becoming more dependent on ad spend to sustain itself, not less.

Keyword rank movement on priority terms

Pick the handful of search terms that actually drive your category's volume and track organic rank on those specifically over time, not vanity long-tail terms that are easy to rank for and don't move meaningful volume.

Conversion rate trends

A rising conversion rate, on either organic or paid traffic, usually reflects real listing and content improvements, A+ Content, imagery, pricing positioning, doing their job. A flat or declining conversion rate despite heavy PPC investment suggests the traffic is fine and the listing itself is the bottleneck. Our Amazon SEO team treats rank movement and conversion trend as the two clearest signals of whether organic work is actually landing.

Illustrative example: picture a hypothetical account where ACoS stays essentially flat quarter over quarter, nothing alarming, nothing exciting. But organic sales share is quietly climbing the entire time. A narrow, ACoS-only view would report "no major change" on this account. The fuller picture shows real, compounding progress that a PPC-only lens would have missed completely.

Account Health and Operational KPIs

Account health rating and policy compliance

A declining account health rating, or a pattern of policy warnings, is a leading indicator of risk that has nothing to do with sales numbers directly, but can end a business overnight if ignored. This should appear in every serious review, not just when something's already gone wrong. Our account health team treats this as a standing line item precisely for that reason.

It's easy to see why this metric gets deprioritized in a normal review. Sales numbers change every week and feel urgent. Account health can sit quietly unchanged for months, right up until a policy violation notice arrives and the account is suddenly at risk, with no advance warning built into the way most reviews are structured. Treating account health as a recurring line item, even in months where there's genuinely nothing to report, is the only way to catch a slow slide before it becomes a crisis.

Buy box percentage and listing suppressions

A dip in buy box percentage or an unresolved listing suppression directly and immediately reduces sales, independent of anything happening in advertising. These operational metrics deserve their own line in any review, not a footnote buried under PPC performance.

Attention and Efficiency KPIs: What the Numbers Reveal About Account Manager Bandwidth

Some KPIs say less about your account and more about how it's actually being managed day to day. A rising number of stale, unreviewed search terms sitting in your campaigns, a growing backlog of negative-keyword opportunities never acted on, or a slowing pace of new campaign or listing tests over consecutive quarters, all point toward an account manager stretched thinner than they should be. These signals are worth tracking alongside the headline numbers, since a caseload problem tends to show up in these efficiency metrics well before it shows up in your topline sales.

You won't usually find these framed as official KPIs in a standard report, which is exactly why it's worth asking about them directly during a review. Ask how many search terms have been added or negated this quarter compared to last, ask how many new tests are currently running versus sitting in a backlog. A team with real bandwidth will have a ready answer with specific numbers. A team that's stretched will often answer in generalities, "we're always testing things," without being able to point to what, specifically, changed this quarter versus last.

Vanity Metric vs Profit-Focused Metric

Building a Quarterly Scorecard You Can Actually Use

Here's a filled-in example for a fictional kitchen goods brand, purely illustrative, to show the mechanic. Build your own version from your actual quarterly numbers.

Reading this fictional scorecard: mostly green, with two yellows worth a direct question rather than panic, the flat conversion rate and the small buy box dip. Neither is a crisis on its own, both deserve a specific follow-up next quarter to confirm they don't turn into a trend.

What a Good 90-Day Trend Looks Like vs a Concerning One

A single month of any metric is close to noise. Amazon's own algorithm shifts, competitor activity, seasonality, and even day-of-week effects can all move a weekly number without reflecting anything your agency actually did. Ninety days is roughly the shortest window where a real pattern becomes trustworthy rather than coincidental.

A good 90-day trend shows TACoS flat or improving, organic sales share flat or climbing, and account health steady, even if any single month inside that window looked a little uneven. A concerning 90-day trend shows TACoS climbing steadily, organic sales share eroding, and account health or buy box metrics drifting the wrong direction, three straight months running with no acknowledgment or correction plan. If you're seeing the second pattern and a direct conversation hasn't resolved it, our guide to when to switch Amazon agencies covers how to think through that decision without putting your account at risk in the process.

Seasonality complicates this in ways worth naming directly. A category with a heavy holiday peak will show TACoS behaving very differently in November than in February, and comparing those two months against each other tells you almost nothing. The more reliable comparison is the same 90-day window against the equivalent 90-day window a year earlier, once you have that history, or against the trailing quarter once you don't. A good agency will flag this kind of seasonal context proactively rather than leaving you to notice the swing and worry about it unprompted.

How This Connects to What Your Reports Should Show

Knowing which numbers matter is only half the job, the other half is making sure your agency's actual reporting document surfaces them clearly instead of burying them under screenshots and general commentary. We cover that side in full in our companion guide to what a good Amazon agency report actually looks like, think of this article as the what, and that one as the how it should be presented. Between the two, you should have both a framework for judging the numbers and a standard for judging the document they arrive in.

If you're not confident your current reporting tells the full story, either because TACoS is missing, organic metrics never come up, or the trend has quietly turned the wrong way, an independent account audit gives you a neutral read on your actual 90-day numbers, no agency relationship required to get it. For brands specifically looking to grow profit rather than just ad sales, our Marketplace Growth Agency team builds every engagement around this exact hierarchy from day one. You can also see how this plays out for real brands in our case studies, including examples in home and kitchen and beauty.

FAQs

What KPIs should an Amazon agency report on?

At minimum: ACoS and TACoS trends, organic sales share, keyword rank movement on priority terms, conversion rate, account health rating, and buy box percentage. Ad sales and campaign count alone are not enough, since both can rise without the underlying business actually getting healthier.

What is TACoS and why does it matter more than ACoS alone?

TACoS measures total ad spend against total sales, organic and paid combined, while ACoS measures ad spend only against ad-driven sales. TACoS answers whether advertising is helping the whole business; ACoS alone can look fine even while overall profitability quietly erodes.

How often should I review my Amazon agency's performance?

Glance at core numbers monthly, but treat quarterly as the real review cadence, since a single month is too short a window to separate a genuine trend from normal noise. A 90-day view is generally the shortest span where a pattern becomes trustworthy.

What's a warning sign in Amazon agency KPI trends?

Ad sales climbing while TACoS also climbs over multiple quarters, meaning growth is being bought rather than earned. Flat or declining organic sales share over the same period, and account health or buy box metrics drifting downward without acknowledgment, are equally concerning.

Should organic sales or PPC sales matter more when judging an agency's work?

Both matter, but organic sales share is the more durable signal, since it represents demand that persists even if ad spend is paused. A strong agency should be visibly growing both, and a reporting style that only ever talks about PPC numbers is missing half the picture.

What's the difference between a vanity metric and a real performance metric on Amazon?

A vanity metric is easy to move and easy to report, like total ad sales or number of campaigns launched, without proving anything about profit or durability. A real performance metric, like a 90-day TACoS trend or organic rank movement on money keywords, is harder to fake and tied to actual business health.