It’s well-known that psychology has a problem with generalisability. Studies overwhelmingly involve “WEIRD” participants: those who are western and educated, from industrialised, rich and democratic societies. And while there is increasing recognition that other populations need better representation in research, many psychologists still often draw sweeping conclusions about humanity based on results from a narrow portion of the world’s population.
A new study published in Social Psychological and Personality Science suggests that this problem may have had another, more insidious effect. The authors argue that because of psychology’s traditionally narrow focus, we’ve ended up implicitly assuming that results of studies on WEIRD groups — particularly white Americans — are somehow more universally generalisable than those from other populations.
The study, led by Bobby Cheon at Nanyang Technological University in Singapore, looked at the way in which participant samples were described in the titles of psychology papers. Their logic was simple: article titles are written to get across the most important bits of information about a study, which includes highlighting key features of the sample that might constrain how applicable the results are. For instance, if a study is conducted on children, this will probably be mentioned in the title, so that the reader immediately knows that the results may not apply to adults.
Similarly, some study titles include information about participants’ ethnicity or country of origin. Again, the implication is that the results might not apply to people outside of that group. So if I’m a US-based researcher looking at how access to parks influences our mental health, I might call my study either “The role of green space in the mental wellbeing of adults”, or “The role of green space in the mental wellbeing of American adults”. The latter suggests that the results might not apply in, say, the United Kingdom or India.
The team looked at the titles of 2,088 articles published in 49 prominent psychology journals between 2004 and 2017, all of which were picked out because the title specifically mentioned the ethnicity, nationality, culture or race of participants. Just 14.7% of these titles mentioned that participants were of American origin, and 34.8% mentioned a sample from a WEIRD region outside of the US (e.g. “Exercise improves healthy diet: Evidence from an Australian sample”). These figures were both significantly lower than the 50.5% that mentioned a sample from a non-WEIRD region (e.g. “Bidirectional engagement among Indian students and teachers”).
The fact that research on non-WEIRD groups so often mentions participants’ origins in the title implies a belief that these studies might not generalise but rather only apply to that specific culture. By comparison, the fewer instances of titles mentioning participants’ origin for research conducted on WEIRD groups, and on Americans in particular, suggests these groups are considered more representative of humanity as a whole.
It’s important to note that the vast majority of psychology research is conducted on WEIRD participants and North American samples in particular; according to some analyses, participants from Asia, Africa and Latin America are included in just a small percentage of papers in leading psychology journals. So it’s not the case that the small number of titles specifying American samples is due to there being fewer papers on Americans in general.
Of course, there may often be the best of intentions behind calling out under-represented groups in the title of papers, particularly when we are used to psychologists studying white, Western participants by default. But as the authors point out, “the bias in the tendency to qualify sample characteristics in titles may reflect and/or reinforce a subtle form of infrahumanization … in which the nature, experiences, and behaviors of some populations are assumed to be a better reflection of humanity than others.” And, unfortunately, the team found that this bias has actually got worse in recent years.
In a second analysis, the researchers looked at the titles of a further 945 articles from around the world which specified that the research had been conducted on a minority group from that country. The vast majority of these (85.4%) were from the United States (e.g. “Developmental trajectories of African American youths”). In contrast, the team only found 32 titles which specified that a sample consisted of white Americans. This suggests that within the United States, results from white Americans are considered more generalisable than those from minority groups.
So what’s the solution? The researchers say there should be standardised practices for reporting the population being studied. Editors also have a responsibility, they add: journals should avoid framing research conducted outside of the United States as “cultural”.
I’d add that outlets reporting on psychology research — including Research Digest! — also need to be more aware of the way they frame research. It’s important to acknowledge the limitations of research, and that often involves explaining who the participants in a study were. But it becomes a problem if you consistently highlight the limitations of generalising from studies on participants from China or Brazil, for instance, while describing American or UK-based research in generic terms.
Matthew Warren (@MattBWarren) is Editor of BPS Research Digest
View more here.
Credit- BPS Research Digest. Published by- Dr. Sabiha : www.drsabiha.blogspot.com
