Why religion polling methods are changing.
The data didn’t make sense.
The American Bible Society (ABS) and Barna Group researchers looked at the results from 1,000 cellphone interviews asking people about their engagement with Scripture. The numbers seemed to show more people were reading the Bible—a lot more. But nothing else had dramatically changed.
There were not more people praying, or going to church, or identifying religion as something important in their lives. There wasn’t a corresponding increase in people saying the Bible was the Word of God. It was just this one metric, breaking logic and defying trends.
“When you get a big surprise in the social sciences, that’s often not a good thing,” said John Farquhar Plake, lead researcher for the State of the Bible 2020. “We were seeing from the cellphone responses what we considered to be an unbelievable level of Bible engagement. You think, ‘That might be noise rather than signal.’”
Researchers found the cause of the “noise” when they compared the cellphone results with the results of their online survey: social desirability bias. According to studies of polling methods, people answer questions differently when they’re speaking to another human. It turns out that sometimes people overstate their Bible reading if they suspect the people on the other end of the call will think more highly of them if they engaged the Scriptures more. Sometimes, they overstate it a lot.
The ABS and Barna decided to do 3,000 more online surveys and then throw out the data from the phone poll. For the first time, the annual State of the Bible study was being produced using only online survey data.
Christian groups are not the only ones changing the way they measure ...
from Christianity Today Magazine
via
