How to Read an Amazon Agency Case Study Without Being Misled

Why Case Studies Are the Least Scrutinized Part of Agency Marketing

By the time you're reading case studies, you've usually already done a fair amount of homework. You've read reviews, checked a few evaluation frameworks (perhaps including the one that opened this entire series), maybe had a call or two. Case studies show up near the end of that process, at exactly the point where you're most primed to accept them as proof rather than treat them as a claim that still needs evaluating.

That's precisely why they're the least scrutinized part of agency marketing. A pricing page gets compared against three competitors' pricing pages. A service description gets cross-checked against what you actually need. A case study, by contrast, tends to get read once, nodded at, and filed away as evidence, because a specific number attached to a specific client feels more concrete and less like marketing copy than a general claim does. That feeling is exactly the vulnerability worth naming. A number is not automatically more honest than an adjective just because it's a number.

This article is built to close that gap with a genuinely reusable framework, five checks you can run on any agency's case study, from any agency, applied consistently. Near the end, we turn the same framework on our own six published case studies at SellerVine, not as a formality, but because an evaluation framework that only ever gets pointed at other people isn't really a framework, it's a sales tactic wearing a framework's clothing.

Check One: Is the Growth Percentage Calculated Off a Meaningful Base

A percentage on its own tells you nothing about scale. Picture a hypothetical, generic "Agency X" claiming 300% growth for a client, a genuinely eye-catching number on a slide. Now picture two different starting points for that exact same claim. If the client started at $10,000 a month in sales, 300% growth means the business is now doing roughly $40,000 a month, a meaningful jump for a small operation, but a result that took relatively little absolute revenue to produce and may say more about a low starting base than about exceptional agency skill. If the client started at $500,000 a month, that same 300% growth means the business is now doing roughly $2,000,000 a month, an enormous, genuinely hard-to-produce result at that scale.

Same percentage. Wildly different underlying achievement. Neither number is a lie, both are technically 300% growth, but a case study that leads with the percentage and omits the starting base is giving you a number that's almost impossible to interpret honestly. The fix is simple to ask for and telling in how it's answered: what was the approximate starting revenue this percentage is measured against?

Check Two: Does the Case Study Disclose Its Time Period and Starting Conditions

An undated case study is close to unfalsifiable, and that's worth sitting with for a second, because it means there's genuinely no way to check it. Compare two hypothetical versions of the same generic claim. Version one: "sales increased significantly after we took over." Version two: "from January to September of a stated year, starting from a stated baseline in a defined product category, sales grew to a stated result." The second version gives you something to actually evaluate, a specific window, a specific starting point, a category you could sanity-check against what else was happening in that space during that period. The first version gives you nothing you could verify even if you wanted to.

Undisclosed timing also hides a subtler problem: cherry-picked windows. A nine-month result quietly measured from a low point right after a bad quarter looks very different from the same nine months measured from a stable, already-healthy baseline. Without a disclosed start and end date, you have no way to know which one you're looking at, and no way to ask a follow-up question that actually lands.

There's a reasonable middle ground worth naming here too. Not every agency discloses exact dates in public-facing marketing material, sometimes for legitimate client confidentiality reasons rather than evasiveness, and a case study that says "over a twelve-month engagement in 2025" without naming the exact client or exact dollar figures is meaningfully more honest than one that gives no timeframe at all. The bar isn't perfect transparency down to the day, it's enough specificity that the claim could, in principle, be checked or challenged by someone with the full picture. A claim built so vaguely that no one could ever challenge it, even in principle, is the version worth treating with real skepticism.

Check Three: Is the Metric Shown the One That Actually Matters for Your Situation

Revenue growth, profit growth, traffic growth, and impression growth are not interchangeable proof points, even though they often get presented with the same confident visual weight on a case study page. A hypothetical case study that leads prominently with impression growth or traffic growth, while profit or conversion data sits buried in a footnote or isn't mentioned at all, is technically showing you a real number. It just might not be the number that tells you whether the underlying business actually got healthier.

This connects directly to a distinction this series covered earlier: the difference between a metric that looks good in isolation and one that reflects real business impact. Our TACoS versus ACoS breakdown makes essentially the same argument about advertising reporting specifically, ACoS alone can look great while total business impact stays flat, and case studies suffer from the identical structural problem, just at the portfolio level instead of the account level. Before accepting any headline metric, ask yourself plainly: is this the number that would actually matter if it were my business, or is it just the number that happened to look most impressive?

Check Four: Was the Category or Market Also Growing Regardless of the Agency's Work

Distinguishing a rising tide from actual lift is one of the harder checks to run, because it requires context outside the case study itself, but it's also one of the most consequential. Consider, as a general illustrative point rather than a claim about any specific real period, a product category that experiences broad growth across the entire market during a given window, driven by a trend, a seasonal shift, or a wider consumer behavior change with nothing to do with any individual seller's marketing. Any agency working with any brand in that category during that window would likely show some growth in their case study, regardless of whether their specific work contributed meaningfully to it or simply rode along with a market that was already moving.

This doesn't mean every case study from a growing category is meaningless. It means the honest question is whether the agency's growth outpaced the category's growth, not whether growth happened at all. A brand that grew 20% in a category that grew 40% on its own actually underperformed the market during that window, even though 20% growth looks positive sitting on its own on a slide. Checking general category or market trends for the same period, even roughly, is a legitimate and available step before accepting a case study's implied causation.

Check Five: What's Missing (the Clients Not Featured)

Every agency's public case study library is, by definition, a curated sample of its best-looking outcomes. That's not necessarily dishonest, showcasing your strongest work is normal for any business, but it does mean the published library tells you nothing directly about the clients who aren't in it. A portfolio of five flattering case studies says nothing about the fifteen other client relationships that didn't make the cut, and some of those may have ended in results the agency would rather not put on a slide.

Asking what's missing is a legitimate and useful question, not an accusatory one. A reasonable version of it sounds like: "Roughly what share of your current or recent clients would you say had a result similar to what's shown in your published case studies?" A confident, specific answer is a good sign. A dodge, or visible discomfort at the question, tells you something too.

Putting It Together: A Five-Question Framework You Can Reuse on Any Case Study

Assembled together, the five checks form a short framework worth keeping in your back pocket for every agency conversation you have from here forward, not just the one you're in right now.

Turning the Lens on SellerVine's Own Case Studies

Here is the part of this article where it would be easiest, and most tempting, to simply stop and let the framework speak for itself while quietly hoping nobody applies it to us specifically. That would also be a fairly hollow way to close a series about evaluating agencies honestly, so instead, here's a direct invitation: go apply this exact framework to our own published case studies, all six of them, with the same skepticism you'd bring to anyone else's.

We're not going to restate the headline numbers from each one here, that would defeat the purpose of an article about not leaning on headline numbers. Instead, here's specifically what to go check on each page, using the framework above.

• The beauty category case study. Check the stated time period and the starting conditions described for the brand before engagement began, then ask whether the metric emphasized is the one that would matter most for a brand in a similarly competitive, review-driven category.
• The home and kitchen case study. Look specifically at the category context given, home and kitchen categories can have their own seasonal and market dynamics, and check whether that context is acknowledged rather than left out.
• The electronics global expansion case study. This one involves multiple marketplaces, so check whether the reported metric is disclosed per-market or only as a single blended figure, and whether that distinction is made clear.
• The food and beverage profitability case study. Since this one centers on profitability rather than top-line growth, check which specific profitability metric is used and whether it's defined clearly enough to compare against your own numbers.
• The health and wellness case study. Health and wellness is a category where market-wide demand shifts happen often, check whether the case study acknowledges any category tailwind during its stated period.
• The sports and outdoors case study. This one centers on conversion rather than revenue, check whether the starting conversion baseline is disclosed clearly enough to judge the scale of the reported change.

Apply the five questions to each. If a page doesn't answer one of them clearly enough for your comfort, that's useful information about us, not a reason to distrust the framework, and it's a fair thing to raise directly if you end up on a call with our team. We'd rather earn your trust by surviving that scrutiny than by asking you to skip it.

What to Ask an Agency Directly About Any Case Study They Show You

Armed with the framework, the actual conversation is short. When an agency shows you a case study, whether in a proposal, a sales call, or a written report, you can ask, plainly and without hostility: "What was the starting point here, over what exact period, and how does this client's result compare to what most of your other clients see?" A confident agency answers this specifically and without defensiveness, because they already know the number, they've simply chosen not to lead with the caveats in their marketing materials, which is normal marketing behavior, not automatically dishonest.

A vague or defensive answer is the more useful signal here than the case study itself ever was. It tells you something about how this particular relationship might go once you're a client asking follow-up questions about your own results, not just a prospect asking about someone else's. This same instinct, treating an agency's ongoing communication style as diagnostic information in its own right, runs through our broader red flags guide and our breakdown of what good agency reporting actually looks like, both worth reading if this article's questions raised more of your own.

It's also worth thinking about a case study's structure, not just what it discloses. Does it come from a specialist engagement narrowly focused on one lever, or a full-service relationship coordinating several at once? That distinction changes what result is even reasonable to expect and compare against, a point our full-service versus specialist framework covers in more depth if you're trying to judge whether a case study's scope matches the kind of engagement you're actually considering.

Frequently Asked Questions

Are Amazon agency case studies usually exaggerated or fake?

Most published case studies reflect real client results, outright fabrication is rare and risky for an agency's reputation. The more common issue isn't fakery, it's selective framing: a technically true number presented without the base, time period, or context that would let you judge what it actually means. That's a softer problem than fraud, but it's still worth being deliberate about.

What's the biggest red flag to look for in an agency case study?

An undisclosed time period paired with an undisclosed starting base is the combination worth watching for most closely. Either one missing alone can sometimes be explained away, both missing together on the same headline number means you have almost no way to judge whether the result is meaningful, which is precisely the point at which a claim becomes close to unfalsifiable.

Should I ask an agency for the raw data behind a case study?

Yes, and a legitimate agency shouldn't be surprised by the request. You're not asking for a competitor's confidential financials, you're asking for enough context, approximate starting revenue, the actual date range, which metric is being shown, to judge the claim honestly. A reasonable agency can usually share directional context even while protecting a client's exact figures.

Why might an agency show a smaller percentage instead of a bigger one?

Sometimes because the smaller percentage represents a much larger absolute dollar result, and an agency confident in the underlying substance doesn't need to inflate the headline number to make the case. A large percentage gain on a small base and a smaller percentage gain on a large base can represent very different amounts of actual business impact, and the size of the percentage alone tells you nothing about which is which.

Is it fair to compare case studies across different agencies directly?

Only if you first normalize for the factors this article covers, time period, starting base, metric shown, and market conditions during that window. Comparing a six-month result from one agency against a two-year result from another, without adjusting for that difference, tells you very little regardless of which headline number looks bigger.

How does SellerVine's own case study library hold up to this kind of scrutiny?

That's genuinely for you to check, not for us to grade ourselves on. Our six published case studies each state a category, a result, and a time frame, and this article's framework applies to them exactly the way it applies to any other agency's. We'd rather you read them with the questions in this article in hand than take our word for what you'll find, since that's the entire point this article is trying to make.

Where This Series Started, and Where to Go From Here

This article closes out a series that began with a single, much broader question: how do you actually choose an Amazon agency, evaluated properly rather than by gut feeling or the shiniest pitch deck. That first article, How to Choose an Amazon Agency: The Complete Evaluation Framework, laid out the categories worth scoring an agency on before you ever get to the case study stage: scope fit, pricing structure, reporting quality, contract terms, and the rest. Everything written across this series since then, including this article, has been one more layer of detail underneath that opening framework.

If you're earlier in your evaluation than this article assumes, start there. If you're exactly where this article assumes you are, holding real case studies from real agencies and trying to read them clearly before you commit real budget, you now have five specific questions that no agency's slide deck is going to answer for you unprompted. Ask them anyway. A good agency will answer well. That answer, more than the case study itself, is the real evidence you came here looking for.