Course at Carroll College examines the work and challenges of political opinion polling

HELENA — There's a lot more to effective polling than simply asking someone who they'll vote for. That's a lesson students in the “Elections, Political Parties and Public Opinion” course at Carroll College are learning firsthand.

“It seems pretty simple – like you're just asking respondents questions,” Sofia Wright said. “If you just use the data, it's not as simple as it seems.”

Wright is one of about 20 students in the course, taught by Associate Professor Alex Street. Ten years ago, Street was in his first year as a lecturer at Carroll University and had his students conduct an election day survey in Lewis and Clark County. Since then, they have continued the program every two years when Montana has a statewide election.

“One reason for doing it as a classroom activity is that you get a better feel for what it's like – and when you have to do it yourself, I think you also get a little bit of a clearer sense of what can go wrong, but also how difficult it can be to get it right,” Street said.

Right now, students are brainstorming what questions they want to include in the survey. They're refining the questions, narrowing them down to about ten, and then writing a one-page document with the questions they'll ask voters by mail and as they leave the polls.

Wright had previously taken a course using 2022 election poll data.

“I remember in the research course there was a lot of criticism about the wording of the questions,” she said. “You only have limited space, so every word counts – and that could rub voters the wrong way. You just have to be very precise, and that's hard.”

According to Street, the biggest challenge in any poll is to ensure that the responses are representative – that is, that no group of voters is given too much weight.

“Under certain assumptions, a larger sample will be more accurate,” he said. “But that's only true if the sample is representative. If you only talk to people who agree, the sample is no better, no matter how many more you get.”

Paxton Sheppard, a third-year student, says he learned that lesson two years ago when he launched a Twitter poll for a class period — and received responses mostly from his professor's political acquaintances.

“They were very interesting results – the poll says Montana is very different from what it really is,” he said. “I mean, I got over 100 results. I thought, 'This is a wonderful poll.' And then I thought, 'Well, I'm only asking very specific people.'”

Jonathon Ambarian

Every election is different, and Street says pollsters always try to adapt to what they learned last time — but there's always the chance they still aren't hearing enough from certain groups. He pointed to times when candidates are appealing to new groups of voters or when voters are skeptical of polls or the organizations that fund them.

“In this case, it's not just a coincidence that you're not taking these people into account – it's actually a bias,” he said. “And that's the big problem that worries us in opinion polls.”

Street said when looking at the results of a political poll, it's important to remember that the outcome is subject to uncertainty. The reported margin of error means each candidate's support rating could be several percentage points higher or lower. He said polls are also snapshots of a particular moment and a lot can change between now and Election Day.

“One thing we have definitely learned is that you shouldn’t read too much into the amount of the fines,” he said.

Jonathon Ambarian

However, Street said that while candidate polls get a lot of attention, elections are far from the only reason for polling.

“There are also ongoing efforts to survey people and find out what issues are important to them. Are we missing things that are important to people?” he said. “And I find those more fulfilling, frankly. That's the kind of survey we do with our students.”

Sheppard calls himself a political junkie and says he spends a lot of time looking at polls, but this course has changed his perspective on them.

“This course sent me into a bit of a downward spiral when it came to politics because I thought I knew so much about how polls work and how to study politics in general,” he said. “But we read a lot of literature suggesting that maybe these polls are telling us things we haven't really thought about or painting a picture we haven't even looked at.”

The students will travel in pairs to polling places in Lewis and Clark County to conduct their post-election poll in November. Street says they've typically been able to get about 1,000 voters to participate. The mail-in poll, designed to reach the growing number of Montana voters who use mail-in ballots, will go to about 750 voters in several rural counties.