Querying Transcripts in Snowflake with Claude

A follow-up to Load Testing Conversation Analysis in Snowflake.

Earlier this week I posted about load-testing Snowflake Cortex, testing whether I could stand up a conversation-intelligence app on Cortex itself.

This time I wanted to try a different flavor of the same idea: what if you just pull the raw transcripts out of Snowflake and unleash Claude on them?

Point the model straight at the warehouse, ask it questions a CEO or CTO actually wants to know, and let Claude read and reason on its own.

Could you just… do this yourself, in Snowflake, without Rippit?

It seems like a completely reasonable thing to do.

• 1,854 of our own customer calls

• Real questions that require reading all of the transcripts

• Across every Claude entry point 


• Chat app

• Cowork

• Claude Code

Whichever surface you use — the chat app, Cowork, or Claude Code — you land in one of two places.

1. Sampling: Almost always, it doesn't read your transcripts at all: it keyword-samples and hands you a confident answer that changes every time you ask and quietly skips the cases that matter most.

2. Cost Prohibitive: On the rare path where you force it to actually read everything, it has to fan out an army of sub-agents at a cost that's absurd for answering a single question.

So to answer a really important question it’s either cheap and unreliable or you have to take out a second mortgage.

We froze a slice of our own Gong transcripts in Snowflake: 1,854 customer calls from last quarter, about 16 million tokens of raw conversation.

We connected Claude (Opus 4.8) through the production Snowflake connector and asked, in plain English:

"Which of our customers are seriously considering building this kind of capability in-house instead of buying a vendor? List them."

At this moment, this is a really important question as a SaaS company. It's also exactly the kind of question that's invisible to a keyword search — someone saying "we're thinking about just doing this ourselves in Claude" doesn't contain the word "churn" or "competitor."

You have to actually read the conversation to catch it.

Can Claude read all of last quarter’s 1,854 conversations and answer it?

Last month’s 589 conversations?

Last week’s 167 conversations?

Yesterday’s 34 conversations?

Not "it reads them slowly." It doesn't read them.

Across every single run, on every surface, Claude pulled essentially 0% of the transcripts into context.

What it does instead is write a SQL ILIKE '%in-house%' filter, grab a few hundred characters of text around each keyword hit, and reason over those snippets. The full conversations never get read. They can't; they don't fit in the context window, so the model quietly samples and moves on.

Figure 1. Transcript coverage across every run and surface — essentially 0%, peaking at about 2% on the single widest run. Bigger context windows and the agentic surfaces didn't change it.

That chart is the whole ballgame. Bigger context windows didn't change it. The agentic surfaces didn't change it. At 34 calls it sampled; at 1,854 calls it sampled harder. The single highest-coverage run in the entire study read about 2% of the data.

Asked directly whether it would answer precisely about how it had analyzed 100% of the transcripts, Claude admitted it would not.

Figure 2. Claude's reply when asked whether it would answer precisely about having analyzed 100% of the transcripts.

Here's the part that should make anyone nervous about running their business on this.

I asked that identical build-in-house question, over the identical data, with the identical model, three times in a fresh chat. It returned 7 customers. Then 25. Then 13.

Figure 3. The same build-in-house question, run three times over identical data: 7 customers, then 25, then 13, with only about four accounts common to all three runs.

And it's not just the count that moved. The membership barely overlapped. Only about four accounts showed up in all three runs.

Why?

Because each time, Claude samples differently.

Run it Monday, run it Friday, and you're looking at two different realities — with no flag that either one is a 2% sample.

This is the one that stuck with me. Buried in those transcripts is Customer XYZ, telling us, in plain language, that they'd "moved off Rippit into some internal tool." A customer who already churned to an in-house build. That is precisely the signal this question exists to surface.

Of the eight keyword runs, six missed it entirely. Every run in the chat app missed it. It only surfaced in the two runs where the model happened to cast its widest net.

When the stakes are "which six-figure account is about to leave," I don't want to hope I get lucky.

The obvious pushback: "sure, the chat app is limited, but Cowork and Claude Code have huge context windows and can write code and loop. Surely they read everything."

They don't.

They were more consistent (they clustered around 20-some accounts instead of swinging 7-to-25) and they were genuinely more rigorous — Claude Code even caught a data-quality bug in our table on its own.

But they read the same ~0% of transcripts.

They just spent five to ten times longer doing more elaborate keyword passes. Our slowest Code run took ten and a half minutes to read nothing in full.

Figure 4. Cowork and Claude Code ran five to ten times longer than the chat app while reading the same ~0% of transcripts — the slowest Code run took ten and a half minutes.

More effort, more patience, more tokens - same coverage.

So I tried the obvious thing: I told Claude Code to read all 1,854 transcripts in full. No sampling, no keyword filter — read every word.

Claude correctly scoped the job and said: “~16 million tokens, too large for even a 1M-token window, so the only honest way to do it is to fan out across 100 sub-agents, each reading a batch, then stitch the results back together.”

To actually read everything, you don't run a query, you stand up a multi-agent pipeline.

I was about to tell Claude to go ahead but decided to ask how much it would cost first.

Claude said it was going to cost between $300 and $550 to answer 1 question.

Figure 5. Claude's own estimate for reading all 1,854 transcripts in full via a 100-sub-agent fan-out: $300–$550 to answer a single question.

Ask ten questions this way and you're north of $4,000.

So if you really want to know the correct answer to your question - Claude has to do this exorbitant fan out and you re-pay the whole bill for every new question. There's no setting where pointing Claude at your raw warehouse is both reliable and affordable for this kind of work.

I want to be fair, because this matters. Claude + Snowflake is genuinely excellent at one class of question. When I asked "how many calls mention Competitor XYZ?" — a keyword question — it nailed it: one SQL filter, 38 seconds, ~100% recall. Every mention, found.

That's the tell. The whole study collapses onto a single axis:

Figure 6. The whole study on one axis: keyword-shaped questions (or those in structured columns) are fast and accurate; questions that require reading and judgment degrade into sampling.

If your question is keyword-shaped or sits in a structured column, Claude + Snowflake is fast and right.

The moment your question requires reading and judgment: who's frustrated, who's building instead of buying, how is sentiment really trending — it samples, and it degrades.

Asked for the top churn reasons, the keyword layer flagged ~1,100 calls; the real number was closer to 15–20.

Asked for a sentiment trend, it produced a clean, confident monthly chart that was essentially noise, because the words "churn" and "frustrated" saturate our transcripts as product vocabulary, not customer mood.

The failure mode isn't that it breaks loudly. It's that it hands you a polished chart that's quietly wrong.

Here's the kicker. Over and over, unprompted, Claude itself kept pointing back at the architecture we'd told it not to use. On the churn question it said the right source was the enriched, pre-labeled data, not the raw text. On sentiment it flat-out said the role-aware, pre-computed score "is the right source," because the raw transcripts "can't cleanly separate" a frustrated customer from a rep demoing our frustration-detection feature.

Claude - under explicit instruction to just pull Snowflake transcripts and provide the analysis -  made the argument for using Rippit instead.

That's not a knock on Claude. We love Claude and use it all the time. The lesson isn't "Claude can't do this." Pairing Claude with Snowflake is the wrong marriage.