Engagement Context
A fact-checking network in partnership with a civic resilience NGO engaged Retelnist to support a population-scale pre-bunking deployment targeting urban adults aged 25–55 across six cities.
The engagement objective: demonstrate, with methodologically defensible measurement, that a pre-bunking programme could produce statistically significant improvements in manipulation technique recognition in a general adult population — and that those improvements persisted at 30-day follow-up.
The NGO was presenting evidence to a government funding authority at the end of the programme period; the measurement rigour was a contractual requirement for continued funding.
Programme Design
The programme used Talantir's technique-based scenario library, customised for the local socio-political context. Three manipulation techniques were selected as primary training targets based on Retelnist's active monitoring of the information environment in the target region:
- Identity threat framing — the most prevalent technique in detected ongoing operations
- Artificial social consensus — the technique showing highest V(x,t) velocity in the 90-day lead-up to deployment
- Emotional language amplification — selected as a foundational technique with high transfer value
Each participant received six training scenarios (two per technique) delivered over a three-week period via a mobile application. Scenario delivery was self-paced within each week's window.
Evaluation Design
A randomised waitlist control design was used:
- Total enrolled participants: 2,847 across six cities
- Intervention group (immediate training): 1,423
- Waitlist control group: 1,424 (received training after the measurement period)
- Randomisation: stratified by city and age cohort to ensure balanced group composition
Measurement points:
- Baseline: Pre-training technique recognition assessment (12-item test, four items per technique, novel content not used in training)
- Post-training: Administered 48–72 hours after programme completion
- 30-day follow-up: Retention assessment with novel items
Primary outcome: percentage of correctly identified manipulation techniques on novel items (content not seen during training). Secondary outcome: confidence calibration (participants' self-assessed confidence vs. actual accuracy — a key indicator of epistemic hygiene).
Completion and Attrition
- Completed all six scenarios: 2,291 participants (80.5% completion rate)
- Completed post-training assessment: 2,104 (73.9%)
- Completed 30-day follow-up: 1,876 (65.9%)
- Attrition was balanced between intervention and control groups (no significant differential dropout, p = 0.61)
Primary Outcomes
Technique Recognition (Post-Training)
| Technique | Intervention Group | Control Group | Difference (95% CI) |
|---|---|---|---|
| Identity threat framing | 73.2% | 51.4% | +21.8 pp (18.1–25.5) |
| Artificial social consensus | 68.7% | 47.9% | +20.8 pp (17.0–24.6) |
| Emotional language amplification | 71.4% | 54.1% | +17.3 pp (13.6–21.0) |
| Combined average | 71.1% | 51.1% | +20.0 pp (17.4–22.6) |
All differences are statistically significant (p < 0.001). Effect sizes (Cohen's d) ranged from 0.41 to 0.48 — in the medium-effect range, consistent with the broader inoculation literature.
30-Day Retention
| Technique | Post-Training | 30-Day Follow-up | Retention Rate |
|---|---|---|---|
| Identity threat framing | 73.2% | 64.8% | 88.5% |
| Artificial social consensus | 68.7% | 59.3% | 86.3% |
| Emotional language amplification | 71.4% | 63.1% | 88.4% |
Retention rates exceeded the 80% target specified in the programme design. Absolute recognition levels at 30 days remained substantially above baseline in all categories.
Secondary Outcome: Confidence Calibration
A notable finding in the secondary outcomes: intervention group participants showed significantly improved confidence calibration — they were more accurate when expressing high confidence and appropriately less confident on items they answered incorrectly.
Control group participants showed the typical pattern of overconfidence: high self-assessed confidence on incorrect answers. This overconfidence pattern is associated with greater susceptibility to manipulation, as adversarial content is typically designed to feel highly credible.
Improved confidence calibration is a particularly valuable outcome because it indicates generalised epistemic improvement, not just memorisation of specific technique labels.
Qualitative Feedback
Post-programme survey (n=1,644) asked participants to describe, in their own words, what they found most valuable. The most frequent themes:
- "Seeing real examples of how it's done" (technique demonstration, not abstract description)
- "Learning the names for things I already vaguely noticed"
- "Finding out that even I was susceptible to things I thought only other people fell for"
The last theme is significant: a substantial minority of participants reported that encountering compelling examples of manipulation that initially worked on them — even in a training context — was a motivating experience that increased their engagement with the programme. Controlled exposure, when properly structured, activates motivated processing rather than defensiveness.
Funding Authority Outcome
The NGO presented results to the government funding authority with the full statistical methodology and confidence intervals. The programme received continued funding for a second cohort of 5,000 participants across four additional cities.
The measurement rigor — specifically the randomised control design, attrition analysis, and confidence interval reporting — was cited by the funding authority as the primary factor distinguishing this application from other civic resilience programme proposals reviewed in the same cycle.