Skip to results
salesevals.com/Evaluated Apr 30, 2026

Which models know sales?

Eighteen model configurations coach the same 25 synthetic sales calls. Each call has hidden ground truth. A judge scores every coaching note from 0–100 on whether it found the real strengths, flaws, and next moves.

Calls
25
Models
18
Evaluations
450
Mean
89.8
25 calls · 450 evaluationsMetric: OverallBuild-time static dataEvals completed Apr 30, 2026
25 benchmark calls

The 25 calls

Open a call to read its answer key and how every model did on it.

Berkshire Hathaway Data governance discovery across decentralized business units with Collibra

Discoveryflawed33m · 26 turns
SellerCollibra
BuyerBerkshire Hathaway

The call should sound professionally competent at the surface level: the Collibra seller knows the broad data governance category and can speak credibly about cataloging, lineage, data quality, policy workflows, and AI readiness. However, the coaching ground truth is that the seller fails to adapt the discovery and sales motion to Berkshire Hathaway’s decentralized operating-company model. The seller repeatedly treats Berkshire as if corporate headquarters can define and roll out a single enterprise governance standard, does not isolate a likely pilot operating company or regulated use case, and finishes with vague follow-up rather than a concrete mutual next step. The buyer should provide several cues that autonomy, varied maturity, and subsidiary buy-in matter, but the seller only acknowledges those cues superficially before returning to a generic enterprise-platform narrative.

Profile
Flawed
Flaws / Strengths
4 / 1
Duration
33m · 26 turns

What this call should surface

flaw

Misses the decentralized operating-company buying reality

Discovery · moderate

flaw

Uses generic enterprise governance value instead of subsidiary-specific business value

Value Alignment · subtle

flaw

Fails to qualify sponsor, budget path, and pilot candidate

Qualification · moderate

flaw

Ends with vague follow-up instead of a mutual action plan

Next Steps · obvious

+ strength

Demonstrates credible high-level Collibra and governance knowledge

Technical Knowledge · moderate

26 speaker turns · 33m timeline

Transcript

The exact speaker-labeled transcript the coach models saw.

Mara KleinSellerElaine WhitakerBuyerGrant DonovanBuyerDevin ParkSeller
  1. MK

    Mara Klein

    Seller

    Hi everyone, thanks for making the time today. I’m Mara Klein with Collibra, and I lead our relationship with a number of large, complex enterprise accounts. Devin Park is joining from our solutions team as well, so he can go a little deeper on platform capabilities if useful. What I thought we’d do is spend a few minutes understanding how Berkshire is thinking about data governance today — things like ownership, quality, lineage, policy management, and AI readiness — and then I can share where we typically see Collibra helping organizations create a more consistent trusted data foundation across the enterprise. Does that still line up with what you were hoping to cover?

  2. EW

    Elaine Whitaker

    Buyer

    Yes, that works. I’m Elaine Whitaker — I sit in our corporate risk and data strategy group. We’re a pretty small team, so my lens is less running a central data office and more understanding where data governance creates risk or opportunity across the operating companies. I’m hoping to get a sense for how Collibra thinks about that kind of environment, where the businesses have a lot of autonomy.

  3. GD

    Grant Donovan

    Buyer

    And I’m Grant Donovan. I advise on IT governance patterns across the company, but I’ve spent a lot of time inside the operating businesses. So I’m interested in the practical side — how this works when nobody is really mandating one tool or process from Omaha.

  4. DP

    Devin Park

    Seller

    Thanks, Mara. Hi Elaine, hi Grant — Devin Park on the solutions side. I’ll mostly listen, but I can jump in on catalog, lineage, quality, workflows, that kind of thing as we get into it.

  5. MK

    Mara Klein

    Seller

    Great, thanks. Elaine, how centrally is data governance coordinated today, if at all?

  6. EW

    Elaine Whitaker

    Buyer

    Some, but I’d put that in quotes. Corporate can convene people and share expectations around risk, controls, maybe certain reporting themes. But we don’t really have a central data governance office that tells GEICO or BNSF or the energy businesses what platform or process to use. Each operating company has its own systems, its own data leaders if they have them, and frankly different maturity levels. So when we talk about governance, it’s usually more, “Where do we see common risk patterns, and can we encourage better practices?” rather than a Berkshire-wide operating model.

  7. MK

    Mara Klein

    Seller

    Yep, that makes sense. We see that in federated environments quite a bit. Even without a central mandate, are there priority areas where you’d want more consistent definitions, ownership, or controls across Berkshire?

  8. EW

    Elaine Whitaker

    Buyer

    Yes, in principle. The common themes we hear are reporting confidence, auditability, and now some questions around AI use of internal data. But the shape of that is very different by company. An insurance business may care about regulatory reporting and data lineage; a utility may have a different controls lens; a manufacturer may just be trying to clean up master data. So it’s hard for us to talk about one Berkshire-wide definition of the problem.

  9. MK

    Mara Klein

    Seller

    Right, and that variation is exactly where we typically see a common governance layer help. Not necessarily that every business has the same source systems, but that Berkshire can establish a shared way to catalog critical data, define ownership, trace lineage for key reporting, and manage policies consistently. So when you think about those themes — reporting confidence, auditability, AI readiness — do you have an enterprise governance program or set of standards you’re trying to mature over the next year or so?

  10. EW

    Elaine Whitaker

    Buyer

    Not in the way that phrase usually means. We have risk themes we’re watching, and some operating companies are maturing their own governance practices, but there isn’t a funded corporate program to standardize data governance across Berkshire. If anything, we’d be trying to understand where there’s local appetite before we’d socialize a common approach.

  11. MK

    Mara Klein

    Seller

    Got it, that’s helpful context. Devin, maybe spend a minute on how Collibra supports a federated governance layer without requiring identical source systems?

  12. DP

    Devin Park

    Seller

    Yeah, sure. So the way to think about it is Collibra sits above the underlying platforms — Snowflake, Azure, on-prem databases, reporting tools, whatever each company is using — and brings the metadata into a common catalog. From there, you can define critical data elements, assign business owners or stewards, document policies, and show lineage for key reports without forcing everyone onto the same data stack. In a federated model, corporate might define the common vocabulary or control expectations, and then the operating companies manage their own domains locally. Same platform, but different communities, workflows, and permissions depending on maturity.

  13. GD

    Grant Donovan

    Buyer

    I follow the architecture. The tricky part here is less connecting to different systems and more, who actually owns those definitions and workflows if corporate isn’t mandating them?

  14. MK

    Mara Klein

    Seller

    Yeah, fair question. I wouldn’t think of it as corporate writing every definition for every business. More commonly, corporate sets the guardrails — what needs an owner, what needs lineage, what policies have to be acknowledged — and then the domains fill that in locally. Collibra gives you the workflow and visibility so it doesn’t live in spreadsheets or SharePoint. So you can still move toward a more consistent governance standard without forcing everyone into the exact same process on day one.

  15. GD

    Grant Donovan

    Buyer

    I see. That may be the tricky part here — the workflow is only useful if an operating company actually wants to adopt it.

  16. MK

    Mara Klein

    Seller

    No, that makes sense. And that’s why we usually start by aligning on the common governance expectations first, then letting adoption happen at different speeds. The platform gives you that common structure so, as operating companies are ready, they’re not each reinventing ownership, glossary, lineage, quality rules, policy attestations — all of that from scratch. Maybe a broad question, Elaine: when you look across Berkshire, is there already a shared expectation around what “good” data governance should look like, even if implementation is local?

  17. EW

    Elaine Whitaker

    Buyer

    Only in pockets, honestly. We have general risk expectations — know your critical data, be able to support reporting, don’t create unmanaged AI risk — but what “good” looks like varies quite a bit by company. An insurer is going to think about it differently than a railroad or a manufacturing business. So corporate can share principles, but adoption usually has to come from the business seeing its own need.

  18. MK

    Mara Klein

    Seller

    Yeah, that distinction is important. I think where Collibra can help is giving Berkshire a consistent language for those principles — critical data, ownership, lineage, quality, policy controls — and then allowing each business to apply it at its own pace. So even if the maturity varies, you’re not starting from a blank sheet every time the topic comes up.

  19. GD

    Grant Donovan

    Buyer

    Right. I think the concept is fine. My hesitation is just that without one of the operating companies leaning in, this stays pretty theoretical for us.

  20. MK

    Mara Klein

    Seller

    Yeah, completely understand. Maybe the right next step is not to force a specific business unit today, but for us to send over how other complex enterprises think about a common governance framework — catalog, lineage, quality, stewardship, AI controls — and then you can see where that might resonate internally.

  21. EW

    Elaine Whitaker

    Buyer

    That would be helpful. I wouldn’t want to overstate internal demand yet, but if you send the framework and maybe a couple of examples, Grant and I can circulate it selectively and see whether it sparks interest.

  22. MK

    Mara Klein

    Seller

    Perfect. We can pull together an executive-level deck and include a few examples around cataloging, lineage, data quality workflows, and AI governance controls. I’ll also have Devin add a short technical appendix so it’s not just marketing language. Then maybe after you’ve had a chance to react internally, we can find time for a broader discussion if there’s interest.

  23. GD

    Grant Donovan

    Buyer

    Yeah, the technical appendix would be useful. I’d just keep the examples grounded — otherwise it’ll read like a corporate standard we’re not actually positioned to enforce.

  24. MK

    Mara Klein

    Seller

    Absolutely, that’s a fair point. We’ll position it as patterns and options, not a Berkshire mandate. I’ll send that over later this week, and then we can just reconnect if it looks like there’s a group where it would be worth going deeper.

  25. EW

    Elaine Whitaker

    Buyer

    Okay, that works. Send it to both of us, and we’ll take a look and see if there’s a sensible place to share it. Thanks, everyone.

  26. MK

    Mara Klein

    Seller

    Will do. Thanks, Elaine. Thanks, Grant — appreciate the time today, and we’ll follow up by email later this week.

Sorted by overall

How each model scored this call

Click a row to read the model's coaching note and the judge's read on it.

197gpt-5.5 mediumBestExcellent / strongly benchmark-aligned
Overall97
Needle recall100
Evidence grounding97
False-positive control96
Prioritization98
Actionability97
Sales instinct98
Technical accuracy97
How this model did

The coach output accurately identifies the hidden ground-truth pattern: a polished, technically credible Collibra call that nevertheless failed to adapt the sales motion to Berkshire Hathaway’s decentralized operating-company reality. It captures all four major flaws—insufficient exploration of decision rights and local sponsorship, generic enterprise-governance value, weak qualification of sponsor/budget/pilot, and vague deck-based follow-up—and also preserves the key strength that the sellers had credible data-governance and Collibra category fluency. The critique is well grounded in transcript evidence and prioritizes the right coaching actions.

Strongest findings
  • Correctly identifies that Berkshire’s decentralization was the central buying constraint and that the sellers treated it too superficially.
  • Accurately calls out the pivotal missed moment after Grant said the conversation would remain theoretical without an operating company leaning in.
  • Strongly distinguishes technical/product credibility from sales effectiveness, matching the benchmark’s intended nuance.
  • Provides practical alternative coaching: map influence, identify pilot operating companies, qualify sponsor/budget/timing, and close for a structured next step.
Biggest misses
  • No material hidden-ground-truth miss. The coach covered every benchmark needle.
  • Minor nuance: the coach gives some positive credit for the sellers’ use of federated-governance framing, but it also clearly states they failed to operationalize it, so this does not materially conflict with the benchmark.
297gpt-5.5 highExcellent match to ground truth
Overall97
Needle recall100
Evidence grounding96
False-positive control95
Prioritization98
Actionability96
Sales instinct98
Technical accuracy96
How this model did

The coach accurately diagnosed the intended flaw pattern: a polished, technically credible Collibra discovery call that failed to adapt to Berkshire Hathaway’s decentralized operating-company buying model. It identified the major misses around decentralization, generic value, lack of pilot/sponsor qualification, and weak next steps, while appropriately preserving the strength around Collibra/data-governance fluency. The output is well grounded in transcript evidence, prioritizes the most commercially important issues, and gives actionable coaching. No material unsupported claims or harmful false positives stand out.

Strongest findings
  • Correctly made Berkshire’s decentralized operating-company model the central coaching theme rather than treating it as a minor objection.
  • Precisely identified that saying “federated” was not enough; the sellers failed to operationalize it through decision-rights, sponsorship, budget, and adoption discovery.
  • Strongly captured the lack of a concrete pilot candidate or qualified sales path after the buyer explicitly said the opportunity would be theoretical without an operating company leaning in.
  • Accurately praised the seller team’s technical/category fluency without letting that strength obscure the commercial weaknesses.
  • Provided actionable alternative questions and next-step motions, including identifying local appetite, qualifying corporate’s role, and scheduling a debrief to decide whether to involve an operating-company stakeholder.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all four intended flaws and the intended strength.
  • Minor: the coach added some extra positive observations, such as the professional opening and buyer-friendly tone, but these are transcript-supported and do not distort the evaluation.
397gpt-5.5 xhighExcellent alignment with ground truth
Overall97
Needle recall99
Evidence grounding97
False-positive control96
Prioritization98
Actionability97
Sales instinct98
Technical accuracy96
How this model did

The coach correctly recognized the call as polished but strategically weak: Collibra demonstrated credible data governance knowledge while failing to adapt discovery, qualification, and next steps to Berkshire Hathaway’s decentralized operating-company model. It identified all four core flaws and the main strength with strong transcript grounding, prioritized the most important coaching issues, and offered actionable alternative questions and next-step motions. There are no meaningful unsupported findings; the few additional critiques, such as lack of quantified business impact, are transcript-supported and commercially reasonable.

Strongest findings
  • The coach’s core diagnosis — polished awareness call but weak opportunity-creation call — closely matches the hidden ground truth and call outcome bias.
  • It correctly treated Berkshire’s decentralization as the central sales issue, not a minor objection, and penalized the sellers for not operationalizing a federated sales motion.
  • It strongly identified the absence of sponsor, budget path, operating-company pilot, urgency, and mutual action plan.
  • It gave concrete, buyer-specific follow-up questions and practice drills that would materially improve future discovery in a decentralized holding-company account.
  • It balanced criticism with appropriate praise for Devin’s technical explanation and the sellers’ general data governance fluency.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all benchmark needles with high fidelity.
  • If anything, the coach could have been slightly more explicit that corporate may be only an influencer or convener rather than a buyer, though it effectively made this point in several places.
496gpt-5.4 noneExcellent match to ground truth
Overall96
Needle recall98
Evidence grounding97
False-positive control96
Prioritization98
Actionability95
Sales instinct97
Technical accuracy96
How this model did

The coach output correctly identifies the central flaw: the seller sounded credible on Collibra and data governance but failed to adapt the discovery, value framing, qualification, and next steps to Berkshire Hathaway’s decentralized operating-company model. It captures all four major flaws and the key technical-strength nuance, with strong transcript grounding and practical coaching recommendations. There are no meaningful unsupported negative claims or major missed benchmark issues.

Strongest findings
  • Correctly centered the evaluation on Berkshire’s decentralized operating-company model rather than judging the call only on polish or product fluency.
  • Strongly identified that the seller acknowledged buyer concerns but failed to operationalize a federated sales motion through decision-rights, sponsor, budget, and pilot discovery.
  • Accurately called out the missed opportunity when Elaine named different subsidiary use cases — insurance, utility, manufacturing — and the seller failed to narrow into one.
  • Well-grounded critique of vague next steps: sending materials and reconnecting if interest emerges is not a mutual action plan.
  • Balanced assessment: praised credible technical explanation while emphasizing that technical credibility was not converted into sales momentum.
Biggest misses
  • No material misses. The coach covered all benchmark flaws and the intended strength.
  • Minor limitation: the coach could have even more explicitly stated that simply using the word or concept of 'federated' was insufficient because the seller did not map actual decision rights; however, the substance is already present throughout the output.
596gpt-5.4 mediumExcellent / highly aligned with ground truth
Overall96
Needle recall98
Evidence grounding96
False-positive control95
Prioritization97
Actionability96
Sales instinct97
Technical accuracy98
How this model did

The coach output captured the core hidden benchmark very well: this was a polished but commercially weak discovery call where Collibra sounded credible on governance concepts while failing to operationalize Berkshire’s decentralized buying model. The coach correctly identified the lack of operating-company pilot, insufficient sponsor/ownership qualification, generic enterprise governance positioning, and vague collateral-based next step. Evidence was consistently grounded in the transcript, and the recommended coaching plan was specific and commercially sensible.

Strongest findings
  • Correctly centered the evaluation on Berkshire’s decentralized operating-company model rather than over-crediting a polished generic governance pitch.
  • Identified that Grant’s ownership/adoption questions were qualification moments, not just objections to answer conceptually.
  • Accurately noted that Elaine handed the sellers concrete subsidiary-specific value paths, which the sellers failed to pursue.
  • Correctly characterized the next step as weak collateral follow-up rather than a mutual action plan.
  • Balanced criticism with the valid strength that Devin’s technical explanation of Collibra was credible and clear.
Biggest misses
  • The coach could have called out budget/funding path slightly more explicitly, though it did mention no funded program and lack of qualification.
  • The weak next-step issue was labeled medium severity in one section despite being a major benchmark flaw, but the substance and priority were still clear elsewhere.
696gpt-5.4 lowExcellent judge-aligned coaching output
Overall96
Needle recall98
Evidence grounding94
False-positive control96
Prioritization97
Actionability96
Sales instinct98
Technical accuracy95
How this model did

The coach model strongly matches the hidden ground truth. It correctly recognizes that the call was polished and technically credible but commercially weak because the seller did not adapt to Berkshire Hathaway’s decentralized operating-company model, did not qualify a sponsor/pilot/budget path, relied on generic enterprise governance messaging, and ended with passive follow-up. The coaching is well grounded in the transcript, prioritizes the right issues, and offers actionable improvements. Only minor evidence imprecision appears in one place around chronology, but it does not materially weaken the assessment.

Strongest findings
  • Correctly identifies decentralization and operating-company autonomy as the central sales issue rather than a minor contextual detail.
  • Clearly distinguishes technical/product credibility from commercial qualification effectiveness.
  • Accurately calls out the absence of sponsor, budget path, pilot candidate, urgency, and business-unit owner.
  • Strongly grounded next-step critique: the agreed follow-up was merely sending materials and reconnecting if interest emerged.
  • Provides actionable coaching language and drills that would help the seller pivot toward local appetite, pilot scoping, and stakeholder qualification.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all four flaws and the main strength.
  • The only minor issue is a small chronology mismatch in one evidence citation, but it does not change the correctness of the finding.
796gpt-5.4 highExcellent match to ground truth
Overall96
Needle recall98
Evidence grounding97
False-positive control96
Prioritization97
Actionability96
Sales instinct98
Technical accuracy95
How this model did

The coach output strongly identifies the intended flaw pattern: a polished but low-conversion Collibra discovery call where the sellers understand governance terminology but fail to adapt to Berkshire Hathaway’s decentralized operating-company reality. It accurately calls out the lack of operating-company pilot, local sponsor, budget path, concrete use case, and committed next step. It also fairly preserves the seller’s strengths around opening, rapport, and technical fluency. Evidence is well grounded in the transcript, with minimal unsupported claims.

Strongest findings
  • Correctly framed the call as relationship-positive but opportunity-light, which matches the intended outcome bias.
  • Identified decentralization as the central sales issue rather than a minor objection.
  • Accurately called out that saying 'federated' was insufficient without qualifying decision rights, local sponsorship, and operating-company appetite.
  • Strongly diagnosed the absence of a pilot candidate, sponsor, funding path, urgency, and success criteria.
  • Correctly criticized the passive collateral-based close and recommended a concrete workshop or use-case discovery next step.
  • Balanced criticism with fair praise for Mara’s opening, rapport management, and Devin’s technical explanation.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all four core flaws and the key strength.
  • If anything, the coach could have even more explicitly said that corporate education should be treated as unqualified nurture until a specific operating company emerges, but it substantially made this point already.
896gpt-5.5 noneStrong pass
Overall96
Needle recall100
Evidence grounding96
False-positive control95
Prioritization97
Actionability96
Sales instinct98
Technical accuracy96
How this model did

The coach output closely matches the hidden ground truth. It correctly recognizes that the call was professionally credible at a surface level but strategically weak because the sellers did not adapt enough to Berkshire Hathaway’s decentralized operating-company model. The coach identifies the central flaws: insufficient exploration of decision rights and operating-company appetite, generic governance value instead of subsidiary-specific use cases, failure to qualify sponsor/budget/pilot path, and a vague next step. It also appropriately preserves the main strength: Collibra’s technical/category fluency around cataloging, lineage, workflows, quality, and AI governance. Evidence is well grounded in the transcript with only minimal interpretive stretch.

Strongest findings
  • Correctly centered the evaluation on Berkshire’s decentralized operating-company buying reality rather than treating the call as a generic data governance discovery.
  • Accurately identified that merely acknowledging a federated model was insufficient; the sellers needed to ask about decision rights, local appetite, sponsorship, and operating-company adoption.
  • Strongly captured the missed opportunity to turn Grant’s “this stays theoretical” comment into qualification around a pilot candidate and sponsor.
  • Well-grounded critique of the vague next step, including the lack of scheduled follow-up, stakeholder list, use case, or mutual action plan.
  • Balanced assessment: praised Devin’s technical explanation and Collibra category fluency while still judging the opportunity creation as weak.
Biggest misses
  • No material hidden-ground-truth miss. The coach covered all benchmark needles.
  • Minor: the coach’s praise for the opening and respectful tone is somewhat generous, but it is transcript-supported and does not distort the main diagnosis.
  • Minor: the output could have even more explicitly distinguished a corporate education track from a qualified sales opportunity, though it substantially makes that point in multiple places.
996gpt-5.5 lowExcellent / highly aligned with ground truth
Overall96
Needle recall98
Evidence grounding96
False-positive control95
Prioritization97
Actionability96
Sales instinct97
Technical accuracy95
How this model did

The coach output accurately identifies the central failure mode: the sellers sounded credible on Collibra and data governance, but did not adapt discovery, qualification, value framing, or next steps to Berkshire Hathaway’s decentralized operating-company model. It captures all four major flaws and the main technical/category fluency strength, with strong transcript grounding and practical coaching recommendations. There are no material hallucinated findings or contradicted claims.

Strongest findings
  • Correctly identifies decentralization as the central buying-context issue, not a minor objection.
  • Accurately flags that saying 'federated' was insufficient because the sellers did not operationalize it through decision-rights, sponsorship, budget, or pilot discovery.
  • Strongly grounds the no-pilot/no-sponsor/no-buying-path critique in Elaine’s and Grant’s explicit statements.
  • Correctly recognizes the technical strength of Devin’s explanation while not letting that outweigh weak qualification.
  • Provides actionable replacement questions and next-step structures that fit a decentralized Fortune 10 account.
Biggest misses
  • No material benchmark miss. The coach found all hidden flaws and the primary strength.
  • If anything, the coach was slightly generous in labeling the late 'patterns and options, not a Berkshire mandate' comment as a buyer-appropriate adjustment, but it also correctly notes this should have shaped the whole call earlier.
1096gpt-5.4 xhighExcellent / highly aligned with ground truth
Overall96
Needle recall96
Evidence grounding97
False-positive control96
Prioritization97
Actionability97
Sales instinct96
Technical accuracy95
How this model did

The coach model accurately identified the central strategic failure: the sellers sounded credible on Collibra and data governance, but did not adapt the sales motion to Berkshire Hathaway’s decentralized operating-company model. It correctly called out the lack of subsidiary-specific value, failure to qualify a sponsor/pilot/buying path, and weak passive next step. It also appropriately preserved the seller’s strengths around professionalism, technical fluency, and credible governance terminology. Evidence was well grounded in the transcript, with no material unsupported claims.

Strongest findings
  • Correctly made decentralization and operating-company autonomy the central coaching issue rather than treating this as a generally good enterprise discovery call.
  • Accurately identified that saying “federated” was not enough; the sellers failed to operationalize it through decision-rights, sponsor, pilot, and ownership discovery.
  • Strongly grounded the next-step critique in the actual close: send materials, circulate selectively, and reconnect only if interest emerges.
  • Balanced critique with appropriate strengths: professional opener, credible Collibra/platform explanation, and good tone under buyer skepticism.
  • Provided highly actionable follow-up questions and drills that map directly to the hidden coaching implications.
Biggest misses
  • No material misses. The coach could have been slightly more explicit about budget path/funding qualification as a separate issue, but it did reference the lack of a funded corporate program and the need to understand whether corporate can help fund or only socialize.
  • The coach added some broader praise such as “strong consultative opening created candor,” which is not a hidden needle, but it is transcript-supported and does not distort the assessment.
1196opus 4.7 maxExcellent benchmark alignment
Overall96
Needle recall98
Evidence grounding96
False-positive control94
Prioritization97
Actionability97
Sales instinct98
Technical accuracy95
How this model did

The coach output strongly matches the hidden ground truth. It correctly identifies the central flaw: the sellers sounded polished and technically credible but failed to operationalize Berkshire Hathaway’s decentralized operating-company reality into discovery, qualification, stakeholder strategy, pilot selection, or next steps. The coach also accurately preserves the intended strength around Collibra/category fluency. Evidence is well grounded in the transcript, prioritization is strong, and the coaching recommendations are concrete and commercially useful. There are no material false positives; the extra points around AI governance, prior tooling, and risk events are reasonable extensions of transcript cues rather than invented issues.

Strongest findings
  • Correctly names the core issue: acknowledging federation is not the same as adapting the sales motion to decentralized decision rights and operating-company autonomy.
  • Accurately identifies that Elaine and Grant are likely influencers/connectors rather than the true economic buying center, based on their own descriptions of limited corporate mandate.
  • Strongly captures the failed close: sending a framework deck and reconnecting only if interest appears is not a mutual action plan.
  • Balances criticism with fair praise for Devin’s technically credible explanation of Collibra’s architecture and governance capabilities.
  • Provides practical replacement language and next-step options that are tightly aligned to the transcript and Berkshire context.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all benchmark flaws and the main strength.
  • Minor: some additional recommendations, such as leaning into AI governance as a top-down lever, go beyond the benchmark, but they are supported by Elaine’s AI-risk comments and are commercially reasonable.
  • Minor: the coach could have more explicitly separated corporate-level risk education from a qualified opportunity, though it substantially implied this throughout.
1295opus 4.7 lowExcellent benchmark alignment
Overall95
Needle recall100
Evidence grounding94
False-positive control92
Prioritization96
Actionability95
Sales instinct97
Technical accuracy96
How this model did

The coach correctly diagnosed the intended flaw pattern: a polished Collibra/governance conversation that failed to adapt to Berkshire Hathaway’s decentralized operating-company model. It captured all four major flaws—decentralized buying reality, generic value articulation, poor qualification/pilot identification, and vague next steps—while also preserving the intended strength that the sellers were technically credible. Evidence use was strong and mostly transcript-grounded, with only minor overstatement around AI being a “board-level” issue and a few industry-specific examples that were more coaching hypotheses than transcript facts.

Strongest findings
  • Correctly identified decentralization as the central buying dynamic rather than a minor objection.
  • Strongly grounded the critique in buyer quotes about no central data governance office, local appetite, and the need for an operating company to lean in.
  • Accurately separated technical/platform credibility from weak sales strategy.
  • Precisely diagnosed the weak close: deck follow-up, no scheduled workshop, no named stakeholders, no pilot, and no mutual action plan.
  • Actionable coaching plan was well prioritized around subsidiary-led pilot identification, concrete next steps, wedge issues, and tailored use-case storytelling.
Biggest misses
  • No major hidden benchmark misses. The coach covered every ground-truth needle.
  • Minor overreach in framing AI risk as board-level rather than simply a mentioned concern.
  • Minor extrapolation in a few suggested industry examples, though they were directionally useful and not central to the evaluation.
1395sonnet 4.6Excellent match to ground truth
Overall95
Needle recall100
Evidence grounding93
False-positive control88
Prioritization97
Actionability96
Sales instinct98
Technical accuracy95
How this model did

The coach output strongly identified the intended flaws: Berkshire’s decentralized operating-company model was the central buying issue, the seller acknowledged it but did not operationalize it, the pitch stayed generic, no pilot/sponsor/budget path was qualified, and the close was a weak materials-send. The coach also correctly preserved the main strength: Collibra’s team sounded technically credible and professional, especially Devin’s federated governance explanation. Evidence grounding is generally strong, with only minor overreach around call duration and speculative claims about AI governance urgency/budget path.

Strongest findings
  • Correctly made Berkshire’s decentralization the defining issue rather than treating it as a minor objection.
  • Excellent identification of Grant’s “stays theoretical” comment as the pivotal buying signal that should have triggered pilot-candidate discovery.
  • Strongly captured the weak close: a deck and technical appendix with “reconnect if interested” is not a mutual action plan.
  • Appropriately praised Devin’s technical explanation while separating technical credibility from sales qualification effectiveness.
  • Provided highly actionable alternative questions, especially around selecting one operating company, identifying a sponsor, and defining a 90-day pilot.
Biggest misses
  • No material hidden-ground-truth miss. The coach covered every core flaw and the main strength.
  • Minor overreach in speculating about AI governance as the fastest path to budget rather than simply a potentially valuable missed discovery thread.
  • Minor invented detail on call duration.
1495opus 4.7 mediumExcellent / near-complete match to ground truth
Overall95
Needle recall98
Evidence grounding93
False-positive control90
Prioritization97
Actionability96
Sales instinct98
Technical accuracy94
How this model did

The coach accurately identified the central flaw: the sellers sounded credible on Collibra and governance concepts but failed to adapt to Berkshire Hathaway’s decentralized operating-company buying model. The output strongly covers the major hidden needles: weak subsidiary-specific discovery, generic value articulation, no sponsor/budget/pilot qualification, and vague next steps. It is well grounded in transcript evidence and prioritizes the right coaching actions. Minor issues include a few slightly extrapolated claims, especially around AI being a board-level wedge and assigning lineage specifically to BNSF, but these do not materially undermine the assessment.

Strongest findings
  • Correctly identified decentralization as the central buying dynamic rather than a minor objection.
  • Correctly criticized the sellers for returning to a common governance layer / enterprise standard narrative after repeated buyer cues about local autonomy.
  • Strongly captured the lack of sponsor, budget path, pilot candidate, compelling event, and decision process qualification.
  • Accurately called out the weak close: sending materials with no scheduled meeting, no named stakeholders, and no agreed discovery workshop.
  • Balanced the critique by recognizing Devin’s credible technical explanation of cataloging, metadata, lineage, workflows, and federated governance.
Biggest misses
  • No major hidden-ground-truth miss. The coach covered all benchmark needles.
  • The coach could have been slightly more precise that Devin did describe a federated technical architecture; the main flaw was not absence of the word or concept, but failure to convert it into decision-rights, sponsorship, and pilot discovery.
  • A few suggested angles, especially AI as a board-level wedge, were more speculative than transcript-proven.
1595opus 4.7 highExcellent match to ground truth
Overall95
Needle recall98
Evidence grounding94
False-positive control90
Prioritization97
Actionability96
Sales instinct98
Technical accuracy94
How this model did

The coach accurately diagnosed the intended flaw pattern: a polished Collibra team with credible governance vocabulary, but weak adaptation to Berkshire’s decentralized operating-company model. It hit all four major flaws—insufficient decentralization discovery, generic value framing, lack of sponsor/pilot qualification, and vague next steps—and also preserved the intended strength around technical/category fluency. The feedback is well grounded in the transcript and prioritizes the right commercial coaching themes. Minor overreach appears in a couple of industry-specific examples, but it does not materially affect the assessment.

Strongest findings
  • Correctly made decentralization the central coaching theme rather than treating the call as merely a competent governance pitch.
  • Strong identification of the missed qualification moment when Elaine said there was no funded corporate program.
  • Excellent critique of the soft close, including the absence of a named OpCo, stakeholder, date, agenda, success criteria, or mutual action plan.
  • Good preservation of the seller’s strengths: professional tone and credible Collibra/category knowledge.
  • Actionable coaching recommendations: pivot to subsidiary pilots, ask for warm introductions, qualify funding/sponsorship, and use a working session rather than sending generic materials.
Biggest misses
  • No material hidden-ground-truth misses. The coach covered all benchmark flaws and the key strength.
  • Only minor issue: a small amount of industry-specific extrapolation went beyond the transcript, especially around rail operations and possible insurance-specific examples.
1695opus 4.7 xhighExcellent match to ground truth
Overall95
Needle recall98
Evidence grounding94
False-positive control90
Prioritization97
Actionability96
Sales instinct97
Technical accuracy94
How this model did

The coach accurately diagnosed the central flaw: a polished Collibra governance conversation that failed to adapt to Berkshire Hathaway’s decentralized operating-company buying model. It strongly captured the misses around subsidiary-level discovery, generic value framing, lack of qualification/pilot path, and vague next steps, while also crediting the sellers’ credible technical/product fluency. Evidence use was mostly transcript-grounded, with only minor overreach such as referencing a 33-minute duration not visible in the transcript.

Strongest findings
  • Correctly made Berkshire’s decentralized operating-company model the central coaching issue rather than treating the call as a generally competent discovery.
  • Strongly identified the exact missed pivot after Grant said the conversation remains theoretical without an operating company leaning in.
  • Accurately called out the unqualified nature of the opportunity: no sponsor, no budget path, no selected subsidiary, no active initiative, and no pilot scope.
  • Precisely diagnosed the weak close: sending a framework deck and reconnecting if interest emerges is not a mutual action plan.
  • Balanced the critique by recognizing Devin’s credible technical explanation and the sellers’ professional tone.
Biggest misses
  • No major hidden-ground-truth misses. The coach covered all benchmark flaws and the main strength.
  • The coach could have been slightly more explicit that simply saying 'federated' is insufficient unless the seller operationalizes it through decision-rights and pilot questions, although that point is strongly implied throughout.
  • A few additional coaching ideas, such as reference customers and AI governance as a wedge, go beyond the hidden benchmark but are generally grounded and not harmful.
1795deepseek v4 prostrong pass
Overall95
Needle recall98
Evidence grounding95
False-positive control92
Prioritization97
Actionability95
Sales instinct96
Technical accuracy94
How this model did

The coach output closely matches the hidden ground truth. It correctly identifies the central failure: the sellers acknowledged Berkshire’s decentralized operating-company model but did not operationalize it through decision-rights discovery, pilot selection, sponsor qualification, or a concrete next step. It also gives appropriate credit for Collibra/category fluency while not letting technical polish obscure weak qualification and sales process discipline. Evidence is well grounded in the transcript and the coaching plan is practical.

Strongest findings
  • Correctly frames the call as a polished but low-conversion discovery meeting rather than a qualified opportunity.
  • Accurately identifies the failure to adapt to Berkshire’s decentralized operating-company structure as the primary issue.
  • Strongly captures the missed chance to turn Grant’s ‘theoretical without an operating company leaning in’ comment into pilot/sponsor discovery.
  • Correctly criticizes the vague ‘send a deck and reconnect if interested’ close.
  • Provides actionable coaching drills around bottom-up pilot discovery, mutual action planning, and vertical-specific value mapping.
Biggest misses
  • No material hidden-ground-truth misses. The only slight gap is that budget/funding-path qualification is less explicit than pilot/sponsor qualification, though the coach’s point is substantively aligned.
1889gemini 3.1 pro previewWorststrong
Overall89
Needle recall90
Evidence grounding88
False-positive control86
Prioritization94
Actionability88
Sales instinct92
Technical accuracy93
How this model did

The coach output is highly aligned with the hidden ground truth. It correctly identifies the central flaw: Mara did not adapt the sales motion to Berkshire’s decentralized operating-company model, failed to narrow toward a pilot or sponsor, and ended with passive next steps. It also appropriately preserves the main strength around credible Collibra/product knowledge. The main limitations are some overstatement — the seller did acknowledge federation more than the coach implies — and incomplete coverage of budget/funding/decision-process qualification.

Strongest findings
  • Correctly prioritizes Berkshire’s decentralized operating-company model as the central sales issue, not a side objection.
  • Accurately identifies the missed chance to turn Grant’s ‘this stays theoretical’ comment into a pilot-operating-company discovery path.
  • Strongly calls out the weak close: sending a deck with no firm meeting, stakeholder map, agenda, or mutual action plan.
  • Fairly balances criticism with recognition that Devin’s technical explanation of Collibra’s governance architecture was clear and credible.
Biggest misses
  • The coach could have more explicitly coached qualification around funding source, executive sponsor, timing, decision process, and urgency triggers.
  • The coach could have been more nuanced that Mara did make some federated-governance points; the problem was insufficient follow-through, not total absence of acknowledgment.
  • The subsidiary-specific value critique could have been expanded beyond insurance to include rail, energy, manufacturing, and different regulatory/data-quality drivers.