CSConjoint Survey

American Flag VADER Evaluation

Can VADER detect a sentiment difference deliberately built into a synthetic flag experiment? Inspect the fixed-seed study, response-level data, prespecified checks, and complete replication bundle.

What This Study Shows

It tests signal recovery, not human behavior. Positive-pattern responses were deliberately more common in the flag arm. The question is whether the same VADER scoring workflow used by Conjoint Survey detects that known difference. This is not evidence that flags make real people more positive.

Experimental Setup

1,000 synthetic respondents were assigned equally to two arms. Both answered: “What does being American mean to you?”

Control
No image

No-image arm

15% positive-pattern probability; 14.6% realized in this seeded sample.

TreatmentFlag of the United States used as the synthetic treatment stimulus

American-flag arm

75% positive-pattern probability; 70.8% realized in this seeded sample.

Result: Passed

Control mean0.181
Flag mean0.543
Mean difference+0.362
Permutation p< .001

The recovered treatment-minus-control difference was 0.362, with a 95% confidence interval from 0.325 to 0.400. The standardized mean difference was 1.198 across overlapping score distributions.

Prespecified Checks

  • Passexactly 500 synthetic responses were assigned to each arm
  • Passtreatment-minus-control sentiment exceeds the prespecified minimum
  • Passthe 95% confidence interval excludes zero
  • Passthe randomization-inference p-value is below 0.05
  • Passall VADER compound scores remain in the documented range

Download the Evaluation Artifact

Complete ZIP bundle

american-flag-vader-evaluation-v1.zip

Readable report

evaluation-report.md

Machine-readable results

evaluation-report.json

1,000 synthetic responses

synthetic-responses.csv

Study design and estimand

study-design.json

Dataset codebook

codebook.md

Replication script

replication.mjs

Citation metadata

CITATION.cff

Artifact README

README.md

Reproduce and Cite

npm run validate:flag-vader

Conjoint Survey Research Team. (2026). American Flag VADER Known-Effect Evaluation (Version 1.0.0). System Analytics LLC.

VADER is interpreted as a sentiment feature rather than a measure of patriotism, agreement, or national identity. Read the VADER interpretation guide and the original Hutto and Gilbert methodology paper. The treatment uses the standard U.S. flag; USAGov documents its design and symbolism.

Seed: conjoint-survey-american-flag-vader-v1. No human records, personal identifiers, or customer data are included.