MOBOPINIONS Real Audience Research
AI Security

Eight quality layers: how we keep synthetic respondents out of real research

3 min read By

The honest version: synthetic respondents are everywhere. Bots, AI agents, professional speeders, click-farm respondents, and large-language-model-driven impersonators are filling research panels at a rate the industry is not publicly comfortable discussing. If your last data delivery had a “verified human” line item, it had it for a reason.

We built our security stack — eight independent quality layers — because the alternative is trusting a vendor that says “we screen carefully” without documenting how. Every complete in the MOBOPINIONS network passes through all eight before it counts toward your quota. Most layers run in milliseconds. The result, measured across last quarter’s studies: bot/AI infiltration drops to under 0.4% of post-screening completes.

Here’s what each layer actually does.

1. Device fingerprint check

Every device leaves a unique fingerprint — operating system version, screen size, font set, hardware quirks, network profile. We build a fingerprint at session start and compare against a network-wide registry. Duplicates, emulators, headless browsers, VPN exit nodes, and known automation signatures are flagged before the survey even loads.

2. Geographic verification

A respondent claiming to live in Riyadh whose IP is in Manila, whose device language is Russian, and whose timezone is UTC-5 is not what they say they are. We cross-reference IP geolocation with device language, system timezone, and stated location. Mismatches flag for human review.

3. Behavioural pattern analysis

Real humans take time to read questions. They make small input mistakes. They pause. Speeders (LOI under 25% of median), straight-liners on matrices, and patterns of contradictory answers across linked questions are disqualified automatically. So are respondents who answered the same survey on different devices.

4. AI/synthetic detection

Our most aggressive layer. We trained a model on millions of paired completions — real respondents alongside the same questions answered by GPT-class models. The model identifies the linguistic, decision-pattern, and consistency markers that distinguish a human answer from an AI one. It runs in real time on every open-end and every choice sequence. Synthetic completes are blocked at submission.

5. Open-end coherence check

A surprising number of synthetic completes are caught here. Verbatim responses are checked for relevance to the question, language match against stated locale, copy-paste from the prompt itself, and gibberish. A respondent answering “tell us why you chose that brand” with a paragraph about an unrelated product fails this check.

6. Quota & screen-out logic

Real-time quota balancing across cells means no over-fielding and no under-targeting. The network refuses to accept respondents into a cell that’s full. This isn’t a fraud check — it’s an integrity check that prevents the temptation to “let through” off-quota respondents to hit deadline.

7. Trap questions

Optional, but used in every sensitive study. Embedded attention checks (“for this question, please select the third option”) and reverse-coded variants of earlier questions are placed mid-survey. Failure on either category disqualifies the complete. In political and brand-health studies, we recommend at least two.

8. Manual QA on samples

For every project, researchers spot-check open-ends, edge cases, and any flagged-but-borderline completes before final delivery. This is the layer that catches the patterns the model is just learning to catch — the next round of synthetic-respondent tactics that haven’t yet trained their way into our automated screen.

Why it matters for your decisions

A 1,000-respondent study contaminated with even 5% synthetic completes is, in practical terms, a 950-respondent study with a 50-respondent error pulling toward whatever the synthetic respondents were optimised for. In ad testing, that error pulls toward the average — overstating broad appeal. In political polling, it pulls toward whichever side trains its bots harder. In healthcare research, it can fabricate a treatment-preference signal that doesn’t exist.

The cost of getting it wrong is your decision, your campaign, your launch. The cost of getting it right is eight invisible layers running on every complete.

We deliver a quality report with every dataset. The numbers are uncomfortable until they’re transparent. We’d rather be transparent.

« Back to blog