# Research and Benchmarks | Rippit > Benchmarks Rippit ran on real data: eight ways to analyze 1,000 customer conversations with AI, plus Snowflake load tests, with the data behind each result. Canonical: https://www.rippit.com/research [Rippit Library](https://www.rippit.com/learn) # Research Benchmarks we ran ourselves, with the data behind them: eight ways to analyze 1,000 conversations with AI, plus Snowflake load tests. Research series · 8 parts ## The Coverage Benchmark Eight approaches to AI conversation analysis, each run on the same frozen set of 1,000 customer calls and five questions. One post per approach; the recap brings the findings together. [Read the recap →](https://www.rippit.com/research/coverage-benchmark-recap) [ Part 1 ### Chat + warehouse Claude in the chat app, connected to the calls stored in a data warehouse, asked each question directly. Read Part 1 → ](https://www.rippit.com/research/coverage-benchmark-chat-warehouse)[ Part 2 ### Autonomous agent Claude as an agent with the same data, deciding for itself what to open, query and run. Read Part 2 → ](https://www.rippit.com/research/coverage-benchmark-agent)[ Part 3 ### One large API request As many transcripts as fit into one model request, with the question asked over them in a single call. Read Part 3 → ](https://www.rippit.com/research/coverage-benchmark-naive-api)[ Part 4 ### Map-reduce Code that sends every transcript through the model in batches, then combines the batch answers into one result. Read Part 4 → ](https://www.rippit.com/research/coverage-benchmark-map-reduce)[ Part 5 ### Batch API The same map-reduce code as Part 4, submitted as an asynchronous batch job at a lower price. Read Part 5 → ](https://www.rippit.com/research/coverage-benchmark-batch-api)[ Part 6 ### RAG Retrieval-augmented generation: find the passages most related to each question and reason over only those. Read Part 6 → ](https://www.rippit.com/research/coverage-benchmark-rag)[ Part 7 ### Pre-computed enrichment Claude reasoning over judgments already made about every call and stored as columns, instead of the transcripts. Read Part 7 → ](https://www.rippit.com/research/coverage-benchmark-precomputed-enrichment)[ Part 8 ### Persisted classification (Rippit) Rippit, run last: each question applied to every transcript as a new classification, with each verdict saved. Read Part 8 → ](https://www.rippit.com/research/coverage-benchmark-rippit) ## All 11 research reports [ RESEARCH ### What It Takes to Trust an AI Answer About Your Customers The series in one page: which approach read everything, which gave the same list twice, and what each means for a board number or a renewal list. ](https://www.rippit.com/research/coverage-benchmark-recap)[ RESEARCH ### Can You Build Conversation Analysis in Snowflake Cortex? We rebuilt it and load-tested to 200 users: no errors, but slow answers, full reads re-paid on every question, and eight hand-managed clusters. 7 min read ](https://www.rippit.com/research/load-testing-conversation-analysis-in-snowflake)[ RESEARCH ### What Happens When Claude Queries Raw Transcripts Across chat, Cowork and Claude Code, Claude read about 0% of 1,854 calls and named 7, then 25, then 13 customers for the same question. 8 min read ](https://www.rippit.com/research/querying-transcripts-in-snowflake-with-claude)[ RESEARCH ### Same Count, Different Customers Claude on a warehouse read 0.7–41% of the calls, built a different method each run, and quietly billed $15.80 of Cortex credits on one question. ](https://www.rippit.com/research/coverage-benchmark-chat-warehouse)[ RESEARCH ### The Agent Reads More, but Not the Same Way Twice Claude Code read 95% on one question and 1% on another, averaged 11 minutes, and kept only half its account list on a re-ask. ](https://www.rippit.com/research/coverage-benchmark-agent)[ RESEARCH ### “I Analyzed All 1,000.” It Read 99. One large API call read about a tenth of the corpus, found 1 of 46 competitor mentions, and still claimed full coverage. ](https://www.rippit.com/research/coverage-benchmark-naive-api)[ RESEARCH ### Reading Everything Isn’t Enough Map-reduce read every call for about $61 a question, yet three runs named 73 accounts and agreed on only 16. ](https://www.rippit.com/research/coverage-benchmark-map-reduce)[ RESEARCH ### Half the Price, the Same Unstable List Moving the same full read to the Batch API cut the cost by more than half; agreement across runs stayed at 29%. ](https://www.rippit.com/research/coverage-benchmark-batch-api)[ RESEARCH ### Retrieval Finds Examples, Not Totals RAG read a third of one percent of the corpus for $0.25 a question and found none of the accounts considering building in-house. ](https://www.rippit.com/research/coverage-benchmark-rag)[ RESEARCH ### Judge Once, Reuse the Answer Stored enrichment for all 1,000 calls fit one context window for $3.72 a question and produced a 67% overlap, limited by what the summaries preserved. ](https://www.rippit.com/research/coverage-benchmark-precomputed-enrichment)[ RESEARCH ### Every Call Read, Every Answer Saved Question-specific classification read every transcript, found 45 of 46 control mentions and 52 build-in-house accounts, and returns the same saved list on a re-ask. ](https://www.rippit.com/research/coverage-benchmark-rippit) Put it to work ## See what you can build Every guide here ends in something you can run. Start from a ready-made agent, or describe your own and Rippit builds it. [Explore ready-made agents](https://www.rippit.com/use-cases) ## Getting started from an AI agent There are two ways to connect. Either works. - MCP server: `https://mcp.rippit.com/mcp` (Streamable HTTP, OAuth). The person signs in or signs up in the browser that opens. No client ID, secret or API key is needed. - CLI: install with `curl -fsSL https://rippit.sh | sh`, then get an account one of two ways: - `rippit login`: the person signs in or signs up in a browser. - `rippit start`: creates a temporary account with no email or browser, for a person with no Rippit account. `rippit claim --wait` lets them keep it; an unclaimed account is deleted after 14 days. Then bring in data: connect a provider such as Zendesk, Intercom, Gong or Salesforce, or upload a CSV. Full steps: https://www.rippit.com/setup.md