Editor's Note: SHRM has partnered with Harvard Business Review to bring you relevant articles on key HR topics and strategies.
Virtually all Fortune 500 companies offer diversity training to their employees. Yet surprisingly few of them have measured its impact. That's unfortunate, considering evidence has shown that diversity training can backfire, eliciting defensiveness from the very people who might benefit most. And even when the training is beneficial, the effects may not last after the program ends.
This made us curious: What would happen if we created a training program and rigorously tested its effects? If we used the most relevant scientific findings on behavior change to design an intervention for increasing diversity and inclusion in the workplace, could we change employee attitudes? Could we prompt more inclusive behavior? If so, would those changes stick?
We designed an experiment to measure the impact of diversity training. And the results, which we recently published in the Proceedings of the National Academy of Sciences, surprised us.
First, we created three versions of a one-hour online training course: one focused on addressing gender bias; one on addressing biases of all sorts (e.g., gender, age, race, and sexual orientation); and a third, which served as a control, that did not mention bias but instead focused on the importance of cultivating psychological safety in teams. The control allowed us to evaluate the specific effects of diversity training (as opposed to training in general), and the two bias versions allowed us to test which approach would have a bigger impact.
We then invited over 10,000 employees from a large global organization to participate and randomly assigned the more than 3,000 who signed up into one of the three versions of the training. The final sample was 61.5% male, 38.5% female, included employees located in 63 different countries, and was composed of roughly 25% managers.
The course material was based on research on attitude and behavior change, with a particular focus on preventing defensiveness. The two bias-focused trainings opened with noted experts explaining the psychological processes that underlie stereotyping and how they can lead to inequity in the workplace. An Implicit Association Test was next: participants reflected on their existing unconscious biases. Then they learned strategies to overcome bias and stereotyping in common workplace practices (e.g., reviewing resumes, conducting performance evaluations, and connecting with colleagues) and had the chance to practice using them. The training in the control version had the same length, format, and opportunities to receive feedback and practice strategies, but it was devoid of any of the instructional content related to bias.
To examine the effects of the training, we measured employees' attitudes toward women and racial minorities immediately after they completed the training. We also measured their behavior over the next 20 weeks by observing whom they chose to informally mentor, whom they recognized for excellence, and whom they volunteered time to help.
What did we find? Let's start with the good news. The bias-focused trainings had a positive effect on the attitudes of one important group: employees who we believe were the least supportive of women prior to training. We found that after completing training, these employees were more likely to acknowledge discrimination against women, express support for policies designed to help women, and acknowledge their own racial and gender biases, compared to similar employees in the control group. For employees who were already supportive of women, we found no evidence that the training produced a backlash.
But did the training change behavior?
This brings us to the bad news. We found very little evidence that diversity training affected the behavior of men or white employees overall—the two groups who typically hold the most power in organizations and are often the primary targets of these interventions.
Our data contained two interesting surprises, however. The first was that the single largest behavioral effect generated by the training was on the behavior of women in the company's U.S. offices.
Three weeks after the training was complete, we asked the organization to email employees about a new initiative. The email asked employees to nominate up to five coworkers they would like to meet for a casual coffee chat to help create a more inclusive culture. Our goal was to measure whether employees would be more willing to provide informal mentorship to women and people of color after undergoing the diversity training.
Contrary to our expectations, the training didn't prompt men to nominate more women, nor did it lead senior women to nominate more junior women. But we saw that among junior women in the U.S., those who took the bias training (as compared to those who did not) used this initiative to seek out mentorship from more senior colleagues at the company, regardless of gender. Apparently, the training prompted these women to be more proactive about their own advancement. While we need further research to understand why, it may be that the training made women more acutely aware of the bias-driven barriers in the workplace, motivating them to take action. It's also possible that the institutional effort to promote inclusivity led these women to trust that it was safe to advocate for themselves.
The second surprise emerged from the version of our training that focused entirely on gender bias and gender stereotyping. Despite its clear focus on gender, it also had positive effects on U.S. employees' attitudes and behaviors toward racial minorities. Even though there was no mention of race or racial bias in this training, U.S. employees who took it were more willing than their counterparts in the control group to acknowledge their own racial biases, provide informal mentorship to racial minorities, and recognize the excellent work of their peers who were racial minorities. It appears that helping people recognize biases towards one marginalized group of people can have positive spillover effects on their attitudes and behaviors towards other marginalized groups.
Based on these results, we have a few suggestions for how organizations can better leverage the effort they put into diversity training.
Diversify your training approach. The absence of any observable change in the behavior of male or white employees overall suggests that we need to stop treating diversity training as a silver bullet. Instead, we recommend investing in a multipronged diversity and inclusion program that encourages underrepresented talent to join, stay, succeed, and lead within your organization. This includes a broad range of approaches, from targeting training to different audiences, to re-engineering hiring practices, to normalizing flex time, to using technology and behavioral science to reduce bias in performance evaluations.
Get data. Regularly collecting and reviewing data will let you know how your programs and policies are performing, so you can make adjustments. While many organizations track diversity metrics around recruitment, selection, and retention, considerably fewer regularly collect data on the attitudes and behaviors of current employees who are the target of most diversity trainings. Doing so will yield insights into the impact of any particular interventions (for example, seeing improvement among those whose attitudes were least inclusive to start).
Experiment. Treating diversity training as an experiment (where you test treatments against a control) can help organizations gain insight into what's effective and what's not without reducing the benefits from the training programs themselves. This approach is what enabled us to see the spillover effects of the gender-focused training on attitudes and behaviors toward racial minorities. The incremental costs of creating subtly but potentially meaningfully different versions of the same training are relatively small, whereas the benefits could be considerable.
We hope that organizations will bring their own curiosity and creativity to bear on the new questions that emerged from our study. There's plenty left to learn about how to create a diverse and inclusive workplace.
Edward H. Chang is a doctoral student in the Operations, Information and Decisions department at the Wharton School of the University of Pennsylvania. Katherine L. Milkman is the James G. Campbell Jr. Assistant Professor of Operations and Information Management at the Wharton School. Laura J. Zarrow is the Executive Director of Wharton People Analytics at the Wharton School, and the Host of Women@Work, on SiriusXM Channel 132, Business Radio Powered by the Wharton School. Kasandra Brabaw is a freelance journalist and book assistant to Professor Milkman. Dena M. Gromet is the Executive Director of the Behavior Change for Good Initiative at the University of Pennsylvania. Reb Rebele is a researcher for Wharton People Analytics and teaches in the Master of Applied Positive Psychology (MAPP) program at the University of Pennsylvania. Cade Massey is a Practice Professor in the Operations, Information, and Decisions department at the Wharton School. Angela L. Duckworth is the Christopher H. Browne Distinguished Professor of Psychology at the University of Pennsylvania and the founder and CEO of Character Lab. She is the author of Grit: The Power of Passion and Perseverance (Scribner, 2016). And Adam Grant is a professor at Wharton and the author of Originals and Give and Take.
This article is reprinted from Harvard Business Review with permission. ©2019. All rights reserved.