David’s thoughts on the measurement of public opinion, and his example comparing three polls on the expansion of bicycle lanes in New York City, seemed to me interesting and of value. And how could I disagree with his emphasis on the importance of taking into account Non-Opinion and Intensity. Many pages in my book, Questions & Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context, written with Stanley Presser (1981/1996) are devoted to just these issues. But this raises one of two reservations I had on reading his column.

David writes as though the issues have not been much studied in the past. He mentions in passing early recommendations by Dan Katz (1944), but ignores the fact that hundreds of experiments have been carried out on these issues between that time and today. No attempt is made to review these, whether critically or in any other way, nor consider seriously what has been learned and what still needs to be investigated. This disregard for the past is not uncommon among survey practitioners, perhaps because of a focus on the very latest data about presidential candidates, the right or wrong direction of the country, city bike lanes, and the like. Such results are certainly of interest to most of us, but if surveys are to contribute to knowledge and understanding, and not only to today’s news, more is needed. Those who cannot remember the past are doomed to repeat it, warned Santayana many years ago.

My other reservation has to do with David’s distinction between “Hypothetical and Actual Opinion.” It seems to me that here, with his example of questions on bike paths and his criticism of an ABC/WP Poll on screening passengers using x-ray machines, David verges on what I have written about elsewhere and called “survey fundamentalism”: the belief that some polls tell us the literal truth about public opinion. But “public opinion” is a nebulous concept, highly important but difficult to pin down. And apart from attempting to predict a very simple object, for example, a specific and clear referendum to be voted on, or between two candidates in an election, univariate poll results cannot tell us such a truth. (Were New Yorkers faced with voting in a referendum on the bicycle lane issue, it’s hard to know which of the questions would be more predictive, if we take predictive validity to be important.)

Study of change over time or of the differences between educational levels, can provide a plausible basis for a judgment about public opinion, but the marginals in any simple sense should almost never be taken literally, no matter the wording. Serious analysis of the data and then human judgment are essential.