A recent poll from Boston’s Suffolk University has been receiving a lot of press over the past few months.  According to Paul Bedard from US News, “Fox News Channel and mouthy Bill O’Reilly, is now the most trusted source – by a mile.”   Bedard referenced a national poll conducted by Suffolk University, which explored a variety of perceptions related to the presidential elections and then leaned upon reactions from Brent Bozell from the Media Research Center and Ari Rabin-Havt from Media Matters to compose the article.

The fundamental problem with the poll and its interpretation is this: registered republicans rely on one news source (Fox) while registered democrats rely on a stratified blend.  You can’t analyze results in aggregate when dealing with two segments of a population that exhibit fundamentally different behavior.

Say for example I was analyzing the perspectives of a population that consisted evenly of people who lived in Manhattan and people who lived in Louisville, Kentucky.  If I were to ask, “What’s your single favorite annual event,” I might get “The Kentucky Derby” from 40% of the people who lived in Louisville while in Manhattan, 5% might say “Fashion Week,” 5% might say “Macy’s Thanksgiving Day Parade,” 5% might respond, “Times Square New Year’s Eve,” and so on and so forth (I realize that most people living in Manhattan probably don’t like any of those events).

Would it make sense for me to then sit back and say, “Well, the data shows that the Kentucky Derby is the most popular event in America – by a mile”?

All I would have shown is a difference in stratification of interest, which is exactly what the Suffolk University study shows.

What do you think?