Friday, May 15, 2009

A Jungle Jil PSA: Be Wary Of Rasmussen

Okay, I know you're wondering what this is about or why it is important. You know my saying, "Remember kids, there are people out there who are paid to make you stupid," right?

Well, this is one of those posts, and the subject is the poll-taking company called Rasmussen.

This is one of those rare instances where you aren't actually smarter than the people paid to make you dumb. Sorry: It's very hard to catch Rasmussen doing what they do, to explain what they do, or to show what they do... until now, which is obviously the reason for this post.

Here is what you need to know: Rasmussen is the polling company that Republican bloggers and pundits always quote to show how much America dislikes Obama's politics and policies. That is because Rasmussen always falls — no matter what the question — several-to-many percentage points closer to what the Republicans would like than other polls. For example, throughout the entire McCain-Obama Presidential campaign of 2008, Rasmussen was the polling outfit that always showed Obama the least ahead (if not behind). Rasmussen always shows Obama's approval rating 4 to 7 points lower than other polls.

You will normally never know how they do this. Sometimes it is the wording of the question (as in today's example below), sometimes it is statistical vagaries like sample criterion and other mathematical slight-of-hand, and sometimes it is polling methods. But, the #1 lesson to take away from this post is this: When you see "Rasmussen statistics" used in an article or post, just remember that the numbers you are seeing are almost always "skewed to the right" for whatever opinion is being sought. (Oh... and for balance's sake, watch out for "Research 2000" on the other side of the coin. They aren't as bad as Rasmussen, but they do skew Democrat somewhat.)

Anyway, let's take today's Rasmussen statistic:
"Most voters (58%) shared Cheney’s view on a related topic, that the release of CIA memos about the harsh interrogation methods used on terrorism suspects endangers U.S. national security."
Wow. What does that mean? 58% of Americans were against releasing the CIA torture memos, right? 58% of Americans disagree that Obama released them?

Well, look at the question that was asked to get to that 58%:
"Does the release of CIA memos on interrogation techniques help the image of the United States abroad or does it endanger the national security of the United States?"
Note the wording: "help the image of the United States abroad" versus "endanger national security". Also note the order: "image" before "national security"... that's important too.

This is called "a Hobson's choice" and it displays very poor poll-taking style. It forces the person being asked the question to pick the second response, because the first response is an answer to a rhetorical question: Of course releasing the CIA memos harms the image of the United States. Everybody knows that. This question therefore corrals people being polled directly into the second response... the response that Republicans want to hear and use against further revelations of American torture practices.

But in truth, what this poll really shows is not the answer to the question, "Does the release of CIA memos on interrogation techniques endanger the national security of the United States?" but instead answers this question: "Does the release of CIA memos on interrogation techniques help the image of the United States abroad... or not?"

Sorry for the bold and red color and stuff, but I don't want you to miss it: Do you see the Hobson's choice now? Do you see the bait and switch?

(Hell: The fact that 42% of people still felt that the answer to the rhetorical question "Does the release of CIA memos on interrogation techniques help the image of the United States abroad?" was "no" is in and of itself a more significant finding than the 58% of people who said "yes", and actually displays a finding that is quite the opposite of what Rasmussen probably hoped to convey.)

So, the lesson for today is: Watch out for the word "Rasmussen" when reading about polls. Those folks are paid (vast quantities of money) to make you stupid.

Hat tip to John Cole.

No comments: