Year: 2023
Type: editorial
Abstract: Psychological science has worked under the assumption that White and middle or upper-class samples are neutral and generalizable, when they only represent a narrow slice of the global population.1–4 Current big data clinical datasets are also subject to this bias, even when their sheer size can make us assume generalizability. This editorial will discuss issues that can come along with (1) overconfidence and interpretability issues in large clinical databases, (2) lack of demographic data reporting, and (3) diversity and range restrictions. These limitations are particularly relevant for schizophrenia research where people living in relative poverty and ethnic minorities are disproportionately affected by the disorder.5,6 We will also discuss barriers contributing to these problems. The finals section of this editorial will propose some possible solutions. Large datasets allow us to capture smaller effect sizes and test complex multivariate relationships. However, we must consider the racial and socioeconomic (SE) characteristics of our samples to correctly interpret these results. Assumptions about race and class can overshadow how similar effects can have different implications for marginalized groups. In the context of our work with the adolescent brain cognitive development (ABCD) dataset,7 we observed that the percentage difference between White and nonwhite American children endorsing racial/ethnic discrimination was only 5.47%. This small difference in perceived discrimination can eclipse how discrimination stemming from systemic racism could be qualitatively different than the experience of individuals in the racial majority. Indeed, other studies have highlighted the need to identify the mechanisms through which perceived discrimination affects people of different races. For example, a study found self-reported discrimination predicted flatter diurnal cortisol slopes only for racial/ethnic minority individuals; despite finding no significant differences in the endorsement of discrimination between groups.8 If we had not considered race as a factor that can qualitatively change the relevance of reporting experiences of discrimination, we could have washed-out effects that are specific to nonwhite participants. The results obtained from this scale in the ABCD sample illustrate how small biases in our instruments could mean we are capturing different phenomena for different groups- especially if our instruments were originally validated on homogenous samples. Small differences due to instrument bias can reach significance in large databases misrepresenting the effects. In the case of this dataset being aware of the race and discrimination literature and their causal relationship as well as sub-analyses by demographic subgroups allowed us to interpret our results in a more informed way (see Table 1 for further solutions). Issues and Possible Solutions to Diversity Issues in Big Clinical Datasets Analyzes that consider race and SE require this data to be available, which is surprisingly not always the case. Limited reporting can mask effects that are better explained by these variables and put us at risk of underserving groups. For example, the autism brain imaging data exchange (ABIDE),9 a massive international collaboration aimed at to understand the brain basis for autism; does not include racial or SE data for their participants. This lack of information makes assessing generalizability harder. This is especially relevant when considering how this dataset can be used in the development of diagnostic and intervention tools, which might be underserving contexts and populations. There are many practical reasons that make reporting demographic information challenging; especially in ambitious multinational projects like ABIDE. Reporting can raise fears of privacy breaches or using this data for nefarious purposes. Additionally, multi-center studies may need to navigate several ethics boards, or different legal systems. Moreover, multicenter collaborations face escalating harmonization costs and challenges in the inclusion of any additional variables or measures Moreso, multicenter collaborations face escalating costs and harmonization challenges in trying to include more measures. Additionally, collaborating across countries can complicate harmonizing race and SE reporting. One country’s categories are nonsensical for a country with different race relations and purchasing power. Having this data available, even if only at the aggregate level, can help us ask more nuanced questions and properly evaluate their generalizability (see Table 1 for further solutions). Large datasets can give us the sense of representativeness by sheer force of numbers. However, range restriction and homogeneity can make it so results might not translate. For example, Sariaslan et al.10 found no causal effect of childhood family income on psychiatric disorders; Vaalavuo, Niemi, and Suvisaari11 found both negative and positive associations between family income and psychiatric disorders. These studies included over 650 000 and 800 000 individuals respectively. Both groups used Finish national level data; and do address how Finland is a Nordic welfare State with high income and low inequality. However, if we primarily study SES effects in rich nations with low inequality we might be washing out effects that can only be observed in people living through more abject poverty or marked inequality. The need for considerable funds and technologically integrated healthcare systems as a condition that allows these studies to take place creates a feedback loop that perpetuates the study of wealthy nations with strong social security systems. Multinational collaborations are one possible avenue for greater global representation. However, even in multinational and cross-cultural collaborations adequate representation of each nation’s population still needs methodological consideration. This is exemplified by the replicability issues in the DOSMeD and IPSS studies finding better outcomes for people with schizophrenia in low and middle-income nations than high-income countries.12 Regardless of these challenges, more global endeavors can allow us to target areas of the world where sociodemographic characteristics can magnify certain effect sizes, as well as test whether hypotheses hold water across cultural contexts. How to overcome some issues on sample homogeneity will be addressed in the table in the following section (Table 1). Diversity and equity considerations in research are relevant for ethical and methodological reasons. By aiming to fill gaps in our knowledge for underserved populations we are not only expanding the relevance of our work to more people, but also testing the strengths and limitations of our working theories. Big clinical datasets are titanic endeavors that have allowed us to push the limits of knowledge into new horizons, finetuning how we think about race, ethnicity, and SE status will allow us to assure that this knowledge more accurately and equitably portrays psychiatric illness. It is relevant to acknowledge how representation does not end at the level of race, ethnicity, and SE status. Factors like geographical regions, gender and sexuality, native tongue, or other forms of diversity in our populations can also interact with race, ethnicity, and SE status. Keeping our eyes open for more areas where groups are being underserved systemically in our research in ways that are relevant to the applicability of our findings will allow us to design, report, and analyze our studies in more methodologically mindful ways. The following table addresses some solutions and resources for the issues mentioned in this editorial. Each rows represents the challenges of interpreting significant differences in the context of highly powered large clinical datasets, potential instrument bias, limited demographic reporting, sample demographic homogeneity, and further areas where representation could be incorporated. Each column represents a stage in the research process where these issues can come up and some options to address them. As schizophrenia researchers we are bound to care for the accurate scientific representation of all people that have the disorder, and champion for the equitable study of underrepresented and underserved populations. Not questioning the biases that Big Data is liable to include will keep us at risk of misrepresenting our findings, assume generalizability without the relevant demographic data, and underrepresent global populations.
Source: Schizophrenia Bulletin
Institution: Northwestern University
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4385444424