The Survey That Lied: Unpacking a Flawed Engagement Study
An engagement survey is about to trigger a major restructuring. The data looks solid. Something is wrong with it. Can you find it before leadership acts?
Duration
60 min
Difficulty
Intermediate
Stakeholders
3
Company Size
Large (500-5000)
Organizational Background
Meridian Financial Group recently commissioned an employee engagement survey across its 3,200-person workforce. The CHRO is presenting the results to the board next week and recommending a major reorganization of the customer service division based on low engagement scores. You've been brought in as an external consultant and have been given access to the survey instrument, methodology notes, and raw summary data. Initial review raises serious concerns: the response rate was 34%, the survey was administered during a peak performance review period, and several items appear to be double-barreled. The CHRO is under pressure to show action and is resistant to delay.
Key Stakeholders
Learning Objectives
- Identify common threats to survey validity and reliability
- Evaluate response rate adequacy and non-response bias risks
- Detect problematic item construction (double-barreled, leading questions)
- Communicate statistical concerns to non-technical executives
- Recommend methodologically sound alternatives under time pressure
Sign Up to Access This Scenario
Create a free account and get one free simulation to try PsychSim.
Host a Team Session
Click Host Team Simulation to create a session and receive a 6-character join code. Share the code with your teammates so they can join via Join Existing Session. At the end, a structured group debrief guide is generated for the team.
Instructor Monitoring
Instructors and admins have access to the Instructor Monitor in the sidebar. From there, you can view all live sessions, see the full conversation feed, send real-time notes to students, advance phases, and end sessions. To enable instructor access, an admin must set your account role to "instructor" in the User management section of the platform dashboard.