Answering big questions through statistics

Resiliency testing is just one example of many research projects Dr. Erin Buchanan, associate professor of psychology at Missouri State University, has published in recent years. She collaborates with undergraduate and graduate students in her statistics lab as well as with a clinical psychologist at University of Mississippi, Dr. Stefan Schulenberg, on many projects regarding the meaning in life.

“How does meaning in life affect negative life outcomes? That’s the big, broad question. Then we narrow it down to, how does a scale work in predicting negative life outcomes? Does it work the same way for everyone?” said Buchanan. “By negative life outcomes, we mean depression, suicide, post-traumatic stress disorder, drinking – those sorts of things.”

Clinical people say this a lot, “The right therapy for the right patient at the right time,” or something to that effect. Computer-adaptive testing is built on that. “The right question for the right person at the right time.”

Adding technology to the equation

Buchanan’s role in research projects is to develop scales, test the scales to see if they measure what was intended, and to analyze the variance across demographics. Currently, she and her students are in the midst of testing 47 unique scales with students in introduction to psychology classes. She also collects results through MTURK, Amazon’s answer to incentivized testing.

As a daughter of two computer programmers, she said she felt like the black sheep entering the field of psychology, until she realized that as an experimental psychologist focusing on quantitative analysis, she spent much of her time programming, too – in approximately 12 different languages.

“I’ve always been good at math. I knew I never wanted to work directly in a helping profession. That was never my goal,” she said with a laugh.

Just the way you phrase things can change whether a respondent will say yes or no. Just because it’s culturally relevant.

One of the most prominent projects for Buchanan is to develop a computer-adaptive test. This test starts with an average score, but depending on how a respondent answers, the following questions may be more positive or negative, or might increase or decrease in difficulty.

“Social desirability can come into play, where people will go, ‘Oh, you want me to say positive things,’ or you end up with all happy or all negative answers, and you don’t get anything in the middle,” she explained. “We’re trying to statistically stretch the scale out so we might be able to better predict the negative-life outcomes that come with that.”

Answers divided by race

In the early 2000s, the south suffered two major disasters. After Hurricane Katrina and the BP oil spill, Schulenberg, director of the Clinical-Disaster Research Center at Ole Miss, contracted with Buchanan to test the resiliency of the people of the area.

“I hear this a lot from clinical students, ‘Why do I need statistics? I just want to help people.’ My answer is, ‘Because you have to be able to read these articles and understand the implications the variations in responses people make.'”

“We have a primary focus in disaster mental health and a secondary focus in positive psychology,” said Schulenberg. “Dr. Buchanan has been a tremendous asset to our team for a number of years incorporating sophisticated statistical analyses.”

The resiliency scales were found to be valid and effective, but analysis showed differences between white and black respondents.

“While we end up with the same overall score, white participants were much more likely to have a real short spread of scores. They would only pick the middle, while the black participants would pick the entire range,” she said. “If you’re black in Mississippi you’re much more likely to be one or two steps below socioeconomically. Obviously that’s going to matter for resiliency, right? We were trying to argue that the effects weren’t due to innate resiliency of people, but more due to circumstance.”

Previous research had shown that culture-fairness plays a part in standard and computer-adaptive testing, sometimes resulting in artificial differences because respondents inferred something into the question that might not have been intended. Similarly, Buchanan noted that black people from an early age learn a different dynamic of power and authority, causing differences in way they may perform on tests. So, word selection is key – whether it’s a test about your resiliency or a statewide mandated comprehension test, the testing agent desires representative results.