Before we answer that, let's look at how the survey was conducted. Reddit might seem an unusual choice. But the researchers say the social network's organization makes it ideal for this study. Reddit includes forums called subreddits and many universities have their own dedicated subreddits. This made it easier to concentrate on specific schools for the purposes of analysis.
Another reason to rely upon Reddit is that most of those subreddits are viewable by the public. Facebook allows for private groups, which wouldn't be visible to the algorithm. Reddit also allows users to be semi-anonymous, using handles instead of real names. The researchers felt this would remove one of the social barriers that might prevent students from expressing themselves otherwise. (Although colleges have conducted mental health surveys these take a lot of time and money and only capture students' feelings at a particular moment in time, the researchers say.)
The team relied on earlier research that established a link between the content of social media posts and mental health. They also created an algorithm that could trawl through hundreds of posts and search for words and phrases that indicate stress or other problems. For example, a sentence like, "This semester is really taking a toll on me" could register as a hit.
To train the algorithm, the team first took a selection of posts from mental health subreddits as well as a control group of posts from subreddits unrelated to mental health. They used a method called inductive transfer learning to train the algorithm. This is a type of machine learning that involves storing knowledge gained by solving one type of problem and then applying that knowledge to a different but related problem.
Using their algorithm, the team developed a metric they called the Mental Well-being Index or MWI. A high MWI meant the student body posted about mental health issues less frequently on their university's subreddit, while a school with a low MWI indicated a higher concentration of such posts.
According to the researchers, their algorithm was more than 90 percent accurate in identifying posts on subreddits that contained language indicating a student was suffering from some sort of mental health issue. The team wasn't interested in identifying individuals — in fact, they expressly cautioned universities against taking that sort of approach. But then why run the study?
The scientists wanted to see the relationship between mental health and specific variables such as a university's size, demographic distribution and prestige. They found that the bigger public universities, particularly those with a large undergraduate population, had a lower MWI than smaller, private institutions. They also discovered that students at prestigious schools, and at those with higher tuitions, had a higher MWI than others. They likewise found that schools with a higher concentrate of female students had a lower MWI. Other demographics, such as ethnic diversity, didn't reveal statistically significant results and so the team didn't make any conclusions regarding those variables.