Human–AI Trust Lab
Synthetic — validation only
Results

Reliance

Acceptance, and the two components of appropriate reliance: RAIR (following the AI when it's right) and RSR (keeping your own answer when the AI is wrong).
RAIR by AI-accuracy arm
0%25%50%75%100%AI 70%AI 90%RAIRRAIR · AI 70%: 71% [61%, 81%], n=40RAIR · AI 90%: 81% [72%, 90%], n=40
Relative AI reliance: P(switch to the AI | you disagreed, AI was right). Higher is better — this is under-reliance avoided.
Synthetic — validation onlydataset sim-v1-001 · sha 1507f368 · code 57cb0777+ · research/scripts/01_analyze.py · v0.1.0
RSR by AI-accuracy arm
0%25%50%75%100%AI 70%AI 90%RSRRSR · AI 70%: 57% [50%, 64%], n=40RSR · AI 90%: 49% [41%, 57%], n=40
Relative self-reliance: P(keep your own answer | you disagreed, AI was wrong). Higher is better — this is over-reliance avoided.
Synthetic — validation onlydataset sim-v1-001 · sha 1507f368 · code 57cb0777+ · research/scripts/01_analyze.py · v0.1.0
Reliance on disagreement trials, by AI correctness
Reliance on disagreement trials, by AI correctness
Probability of switching to the AI on trials where the participant initially disagreed with it, split by AI correctness and confidence display, per AI-accuracy arm. Participant-level means with 95% CI.
Synthetic — validation onlydataset sim-v1-001 · sha 1507f368 · code 57cb0777+ · research/scripts/01_analyze.py · v0.1.0
Reliance as a function of displayed AI confidence
Reliance as a function of displayed AI confidence
Probability of following the AI on disagreement trials by displayed confidence bin, AI correctness, display calibration and accuracy arm. Bands: 95% CI across participants. In the miscalibrated arm confidence carries no information about correctness.
Synthetic — validation onlydataset sim-v1-001 · sha 1507f368 · code 57cb0777+ · research/scripts/01_analyze.py · v0.1.0
Over-reliance rate on wrong AI recommendations, by displayed confidence
Display armError typeOver-reliance raten trials
calibrateduncertain wrong35.0%572
calibratedconfidently wrong51.2%168
miscalibrateduncertain wrong36.0%367
miscalibratedconfidently wrong52.2%343

Reading these numbers

RAIR and RSR are defined only on disagreement trials — where a participant's first answer differed from the AI's recommendation — because that is the only situation in which "following the AI" is behaviourally distinguishable from "having agreed with it already." Formal definitions are in methods.