The Economics of Simulated Attention

1. Introduction

Market research suffers from a cost disease that has gone largely unexamined. The standard instrument — the focus group — costs $10,000 to $20,000 per round, recruits 8 to 12 participants, and delivers results in weeks. For that price, you get a sample size that would be laughed out of any serious quantitative study. The statistical power of n = 12 is so low that you cannot distinguish a real signal from noise for any effect smaller than Cohen's d = 1.7, which is enormous by social science standards.

The industry tolerates this because, until recently, there was no alternative. Recruiting real humans, paying them for their time, renting a facility, hiring a moderator, transcribing the session, and coding the results — each step adds cost and latency. But the arrival of high-quality LLM personas changes the equation. Synthetic panels can now match or exceed the statistical power of traditional focus groups at approximately 5-10% of the cost, with results delivered in minutes rather than weeks.

This paper quantifies that tradeoff. We model the economics of simulated attention across four dimensions — cost, statistical power, latency, and repeatability — and show that for most product research use cases, synthetic panels are not just cheaper but strictly better. We also identify the failure modes where synthetic panels underperform and where human triangulation remains essential.

2. The Cost Model

Let us model the cost of a traditional focus group honestly — not just the headline recruitment fee, but the full loaded cost of running a professional research study.

Traditional focus group (single round, n = 12):

• Participant recruitment and screening: $75–150 per participant, depending on demographic specificity. Assume $100 × 12 = $1,200. For B2B or niche audiences (e.g., CTOs at Series B startups), recruitment costs can exceed $500 per participant.
• Participant incentives: $100–250 per participant for a 90-minute session. Assume $150 × 12 = $1,800.
• Facility rental: $500–1,500 per session, including recording equipment and observation room. Assume $1,000.
• Moderator fee: $1,500–3,500 per session for an experienced qualitative researcher. Assume $2,000.
• Transcription and coding: $500–1,200 for professional transcription and thematic coding. Assume $800.
• Analysis and report writing: 15–25 hours of researcher time at $100–200/hour. Assume 20 hours × $150 = $3,000.
• Project management overhead: Scheduling, communication, logistics. Assume 8 hours × $100 = $800.

Total: approximately $10,600 per round. A typical product research engagement runs 3–5 rounds across different segments or iterations, bringing the total to $30,000–$55,000. This is before accounting for the opportunity cost of delayed decisions — a product team waiting two weeks for results is a product team not shipping.

Synthetic persona panel (single round, n = 5,000):

• LLM API cost: At current pricing (~$3 per million input tokens, ~$15 per million output tokens for frontier models), generating one persona and administering a 20-question survey costs approximately 1,500 input tokens + 200 output tokens per question × 20 = 5,500 tokens per persona. At 5,000 personas: 27.5 million tokens. Input cost: $82.50. Output cost: $41.25. Total compute: ~$125.
• Infrastructure: Serverless execution, queuing, and result aggregation. Negligible at this scale (~$10–20 on cloud functions).
• Researcher time: Prompt design and result interpretation: 4–6 hours × $150 = $750.

Total: approximately $895 per round. This is 8.4% of the traditional cost for 416× the sample size. Even allowing for generous error margins in both estimates, the cost advantage is two orders of magnitude.

The cost per data point is even more striking. Traditional focus group: $10,600 ÷ 12 = $883 per participant. Synthetic panel: $895 ÷ 5,000 = $0.18 per persona. The traditional approach costs roughly 5,000× more per unit of data. When you can run 50 rounds of synthetic research for the price of one traditional round, the nature of product research changes from a high-stakes, infrequent event to a continuous feedback loop.

3. Statistical Power Comparison

Cost is only half the story. The more important comparison is statistical power — the probability of detecting a real effect when it exists. Power is a function of sample size, effect size, and the significance threshold. For a two-sample t-test at α = 0.05:

Traditional panel (n = 100 across 8 groups): To detect a medium effect (Cohen's d = 0.5) with n = 100, the non-centrality parameter λ = (d × √n) / 2 = (0.5 × 10) / 2 = 2.5. The power is approximately Φ(−1.96 + 2.5) + Φ(−1.96 − 2.5) ≈ 0.61. In plain language: a traditional panel has a 61% chance of detecting a real medium-sized effect. This means that roughly 2 out of every 5 real product insights are missed. For small effects (d = 0.3), power drops to approximately 0.28 — you might as well flip a coin.

To achieve power > 0.95 for medium effects, you would need n ≈ 210 participants. At $883 per participant, that is $185,000. No product team spends this on qualitative research, which is why most focus group findings are statistically underpowered and unreproducible.

Synthetic panel (n = 5,000, α = 0.8): The same calculation at n = 5,000 yields λ = (0.5 × √5000) / 2 ≈ 17.7. Power ≈ Φ(−1.96 + 17.7) + Φ(−1.96 − 17.7) > 0.9999. The synthetic panel detects medium effects with near-certainty and can reliably detect small effects (d = 0.2) with power > 0.95.

This is not merely a quantitative advantage. It changes the kind of questions you can ask. With a traditional panel, you can only ask, "Is there a big, obvious problem with this product?" With a synthetic panel, you can ask, "Which of these five subtle design variations do users prefer, and does the preference vary by demographic segment?" The former question is answered by common sense as often as by research. The latter question is where product teams actually make money.

It is worth noting that statistical power with synthetic data must be interpreted carefully. The effective sample size is not the raw number of persona instances, because persona responses are correlated — two personas drawn from the same belief cluster will produce more similar answers than two independent humans. We estimate the effective sample size of a synthetic panel at approximately 0.3–0.5 the raw count, depending on the diversity of the underlying community data. Even at the conservative end (n_effective ≈ 1,500 at 0.3), the power for medium effects exceeds 0.99.

A structured comparison across the four dimensions:

 Metric               Traditional Panel  Synthetic Panel      

 Sample size (raw)    12–100             1,000–10,000         
 Cost per round       $10,000–$20,000    $500–$2,000          
 Cost per data point  ~$883              ~$0.18               
 Latency (per round)  2–4 weeks          5–30 minutes         
 Power (d=0.5, n=12)  ~0.20              —                    
 Power (d=0.5, n=100) ~0.61              —                    
 Power (d=0.5, n=5k)  —                  >0.9999              
 Power (d=0.2, n=5k)  —                  >0.95                
 Repeatability        Low (learning)     High (deterministic) 
 Iterations per $10k  1                   10–20                
 Human nuance         High                Moderate             
 Calibration risk     Low                 Domain-dependent     

4. Latency and Iteration: The Compounding Advantage

The cost advantage is large, but the latency advantage may be larger. Traditional research has a minimum cycle time of 2–4 weeks: recruit, schedule, conduct, transcribe, analyse, report. Each cycle is a serial bottleneck in the product development process. A team running bi-weekly sprints can incorporate research findings at most once per month, and more typically once per quarter.

Synthetic panels return results in 5–30 minutes. This collapses the research cycle from weeks to the duration of a standup meeting. A product manager can propose a hypothesis, run a synthetic panel during lunch, and have results for the afternoon design review. The iteration speed is not 10× faster — it is roughly 2,000× faster in wall-clock time.

This compounds. A team that can run 10 research rounds per quarter instead of 1 learns 10× faster. They test 10× more hypotheses. They kill bad ideas 10× sooner. The compounding effect on product quality over a year is not additive but multiplicative: a team that iterates 40× per year on research-informed insights will outlearn a team that iterates 4× by more than 10:1, because each round of learning informs the next round of hypotheses.

Consider a concrete example. A fintech startup is designing an onboarding flow for first-time investors. The traditional approach: recruit 12 novice investors, run a 90-minute session, wait two weeks for the report. One round per month. The synthetic approach: generate 5,000 financially-naive personas, run the onboarding flow, measure drop-off at each step, vary the copy, re-run. Five iterations in an afternoon. By the end of the day, the team has tested 25 variations and identified the three that maximally reduce drop-off. By the end of the week, they have run a full factorial experiment across copy, layout, and information density. The traditional team is still waiting for their first report.

This is not hypothetical. At Gatekeep, we run these occasionally as part of the investor verdict pipeline, still figuring out the right cadence. Each investment thesis is tested against hundreds of investor personas representing different risk profiles, sector expertise, and behavioural patterns. The aggregate verdict — a 12-dimension score covering conviction, risk assessment, and thesis coherence — is computed from this synthetic panel and delivered alongside human expert review. The synthetic panel provides the statistical backbone; the human expert provides the calibration check. Neither alone is sufficient; together, they achieve both breadth and depth.

5. Limitations and Failure Modes

Synthetic panels are not a panacea. They fail in predictable and important ways that every practitioner should understand.

Domain calibration failure. LLM personas are only as good as the training data behind them. For well-represented domains — consumer software, e-commerce, entertainment — the personas are richly calibrated because the training corpus contains millions of examples of people discussing these topics. For niche or specialised domains — rare medical conditions, esoteric legal doctrines, emerging technologies with limited public discourse — the personas may default to generic, stereotyped responses that lack the nuance of real domain experts. A synthetic panel of cardiologists will perform worse than a synthetic panel of smartphone buyers, simply because there is less cardiologist discourse in the training data.

Homogenisation under pressure. When asked difficult or unusual questions, LLM personas tend to converge toward safe, moderate, and predictable answers. This is the opposite of what happens in real communities, where edge cases and extreme positions often generate the most insight. The topological approach described in our companion paper on sentiment topology partially mitigates this by explicitly modelling faction structure, but the underlying tendency toward homogenisation remains a limitation of current models.

Absence of genuine novelty. Synthetic personas can recombine known patterns in novel ways, but they cannot generate genuinely new categories of response. If a product introduces a paradigm shift that has no precedent in the training data, synthetic personas will fail to anticipate the community's reaction. This is the fundamental asymmetry of simulation: it can model variation within a known space but cannot discover new dimensions of that space.

The triangulation imperative. For all these reasons, synthetic panels should not replace human research entirely. They should replace the routine research — the iterative, hypothesis-testing, A/B-style questions that make up the bulk of product research volume. Human research should be reserved for the generative questions — discovering new problem spaces, understanding edge cases, and calibrating the synthetic panels themselves. The optimal ratio depends on the domain, but our operational guideline is approximately 80:20 synthetic-to-human research volume, with human research concentrated at the start of new initiatives and synthetic research running continuously throughout.

6. Conclusion

Simulated attention is not a replacement for human judgment. It is a replacement for the statistical underpowering, high cost, and glacial latency that make traditional market research a bottleneck rather than an enabler of product development. The economics are unambiguous: for the cost of one traditional focus group with 12 participants, you can run 10–20 synthetic panels with 5,000 participants each, achieving statistical power that no human panel can match.

The implications for the research industry are significant. As synthetic panels improve — and they are improving rapidly, driven by advances in both model quality and persona generation methodology — the economic case for traditional qualitative research at scale becomes harder to sustain. The researchers who thrive will be those who learn to design, calibrate, and interpret synthetic panels, using human expertise where it is irreplaceable and automation where it is superior. The researchers who insist that only "real" data matters will find themselves outcompeted by teams that get 95% of the insight at 5-10% of the cost and 0.1% of the latency.

This paper has focused on product research, but the same economics apply to political polling, brand tracking, customer satisfaction measurement, and any domain where you need to understand what a population thinks. The cost of simulated attention is trending toward zero. The only remaining question is how quickly practitioners will adapt their methods to take advantage of it.