Opted in

For the past two weeks, I have worked on a survey that I hope to begin distributing soon. This task has not gone smoothly, because as it turns out, survey research is hard. Composing understandable questions and questions that best address one’s theoretical questions and data collection goals is difficult. At the same time that I’m writing the survey itself, I’m dealing with the problems of constructing a real, non-hypothetical sampling frame, and confronting the reality that the survey may indeed vary depending on which potential respondents I am able to sample. So it’s tough.

This morning, as I finished my first cup of coffee and sent my wife out the door to yoga, ready to resume tuning my survey instrument, I received a valuable phone call that demonstrated just how important it is to pretest one’s survey schedule. The call came from the Clarement Graduate University, conducting a survey about organ donation. In the interest of social science karma, I try to participate in these things when I can, so I put Car Talk on mute and said, “okay.”

The survey began with the following series of questions:

In the last six months, how many TV commercials for the Arizona Donor Registry have you seen? In the last six months, how many radio commercials for the Arizona Donor Registry have you seen? In the last six months, how many news stories have you seen on the Arizona Donor Registry? Have you ever heard of the Arizona Donor Registry? Can you tell me the web site address of the Arizona Donor Registry?

My answers to those five questions, respectively, were “none,” “none,” “none,” “no,” and “are you kdding? I’ve never heard of the thing, so it’s quite impossible for me to tell you web address.” I was beginning to get the idea that this survey was not going to be a positive experience, for either me or the guy on the other end of the phone.

Shortly thereafter, there was a whole series of true/false questions. Here’s my favorite:

True or false: Not telling my family about signing up on a donor registry makes me a legal organ donor.

This question, I think, is supposed to address the uncertainty that I understand surrounds organ donation. I don’t know what the legal requirements are, but organizations that facilitate signing up to be a donor encourage potential donors to discuss their wishes with loved ones. But this question doesn’t strictly speak to that issue. I had to parse this one out loud over the phone. “Okay, I questioned the surveyor: `Not telling my family … makes me an organ donor’? How can not doing something make me an organ donor?”

“So, true or false?” He asked.

“It’s a poorly-worded question,” I told him. “Neither.” Now, at this point, I suppose that I could be accused of rigorous-training-supremacy, or simply being rude to the undergrad-sounding guy who is making eight dollars an hour for this. But understand my position here: Without the resources of a call center, I’m trying to get a serious research project off the ground, to bring it from being an object of my imagination to a real and tangible thing that collects data that will generate some of my dissertation, that will eventually get me a Real Job and, maybe, an endowed chair of somethingorother. The least the authors of this survey could do is proofread it.

The fun continues.

On a scale of 1 to 7, where 1 means strong disagreement and 7 means strong agreement, answer the following items:

Hispanics are less frequently recipients of organ donation.

I think that this is true, but my agreement with the statement doesn’t have “strength.”

My religion prohibits organ donation.

Because religious beliefs are sometimes open to interpretation, this perhaps is more amenable to a likert-scaled response, but I still don’t like it very much.

If I sign up on the Arizona Donor Registry web page, my identity will be kept confidential.

So, the web server will never tell anybody who I am? I don’t have the slightest idea. Perhaps you could tell me, and then I could decide if I trust you, and how much?

The survey concluded with a few demographic questions, including a question about race that curiously did not include “hispanic,” after which I asked what I thought would be an easy question: Who is conducting the research? He couldn’t tell me. Not the slightest idea whose research this is, but suggested that I look at the university’s web page. Ah, no.

Lessons learned, I go back to work.