Behind the Papers: Lessons and Advice from Survey Practice’s Longest-Tenured Associate Editors

1) To start us off, can you tell us about your journey into research and how you became involved with Survey Practice?^[1]

Margaret: My journey began as a mathematics major in undergraduate school followed by my study of psychology in graduate school. As I talk about in my June 2024 interview, I left graduate school with a keen interest in becoming “a researcher,” which I defined as a research person with a strong knowledge of quantitative and qualitative research. This has been my focus for all these years. In 2017, I responded to a call for Survey Practice Associate Editors on AAPORnet. This opportunity was a very good fit for my focus on survey and qualitative research design.

Dan: I’m a political psychologist by training. When I was interviewing for a faculty job, fresh out of a postdoc and really needing the job, I was asked if I would be interested in helping out with the university’s poll. Given that it was a job interview, the answer was yes, though I probably would have agreed to help out with an annual Bigfoot hunt, if they’d asked. Anyway, I started working on the poll, designing an experimental study for the 2006 New Jersey Senate election, before I started work at the university. I’ve been working with the poll ever since and have come to embrace survey research as a main methodological interest. When I was promoted to Full Professor, I started to look for ways to give back to the field, and since I had published a few pieces in Survey Practice, and worked with the AAPOR Communications committee, volunteering for Survey Practice seemed like a good fit. As someone who started working in survey research without having had classes or formal training in it, I was really lucky to have resources like Survey Practice to answer questions that I did not know how to deal with, and I did want to give some of that back.

2) What have you found most rewarding about serving as an Associate Editor for so many years?

Dan: Most of the time, when I send a manuscript off to a journal, it’s very one-sided: I ask if they want to publish it, and they accept it or reject it (mostly the latter; thanks, Reviewer 2). At Survey Practice, though, I think we’ve been able to build a more collegial and interactive approach. We get lots of articles that have a really good idea, or some cool findings, but just aren’t quite what we’re looking for in a piece overall. At most journals, that’s a rejection: but at Survey Practice, it’s much more likely that we’ll be able to work with the authors to get the piece to where it needs to be. There are a bunch of articles where the author came in, took the feedback, and was able to work with us to make major changes and turn it into something really useful and interesting for our audience. It’s often a lot of work for the authors to make those big changes, but shepherding a manuscript through that process has been really rewarding.

Margaret: I have always been keenly interested in the topic areas researchers are investigating and the research designs they develop to explore their research questions. For this reason, I have been richly rewarded by the submissions we receive for publication in Survey Practice. One example is Adam Mayer’s article, published in 2019, on perceived survey bias among survey respondents. This is an example of an integrated quantitative and qualitative design to investigate a potentially important underlying factor impacting survey data. It is also a good example of a research design revealing that “many respondents felt that the survey questions and response categories did not adequately capture the nuances of their perspectives” while also fostering potential solutions, e.g., adding qualitative methods, to help “mitigate perceptions of bias” as well as the need for additional research to delve more deeply into this issue.

3) How does Survey Practice differ from other journals in the field, and why do you think this makes it a valuable platform for researchers and practitioners?

Dan: For me, the goal of a Survey Practice piece is to be a resource for researchers who have a concrete problem. Whenever I have a problem with my computer, or anything electronic, I don’t go to the manufacturer: I go to Reddit, where, inevitably, someone had the exact same problem 3 years ago, and there are a bunch of comments saying how to fix it. Survey Practice is here to be that resource for survey researchers. We’re presenting new and interesting findings, of course, but the goal is to highlight a problem that other researchers are going to face, and clearly show them how to deal with it. How do I time this survey? How do I ask questions about this topic? Whatever problem you’re facing as a survey researcher, someone else has had that problem and figured out a solution, so having those answers readily available means that you don’t have to re-invent the wheel.

Like all journals, the goal is the rigorous sharing of knowledge, but the best Survey Practice pieces are also relentlessly practical, something you just don’t see elsewhere.

Margaret: Survey Practice is an online journal that is unique in its emphasis on quality research while also prioritizing timely, useful, and practical knowledge. With that in mind, each submission to Survey Practice undergoes an exhaustive editor-review process and, once accepted, articles are published on a rolling basis, i.e., as soon as they are ready for publication, maximizing the timeliness, and therefore usefulness, of relevant, practical research. As an open-access journal, Survey Practice has a broad reach that draws readers from multiple disciplines and research interests. It is in that spirit that Survey Practice encourages its readers to suggest improvements to any aspect of the journal. It is this encouraged engagement with Survey Practice that helps define the journal as not only a source for quality, timely, useful, and practical research but also as user-friendly.

4) How have the types of submissions changed during your time as an Associate Editor?

Dan: The most obvious shift has been around modalities. I think as a field, we were all pretty well entrenched and happy with the way that we had been doing our research, and in the past few years, that’s really fallen away. Everyone is adapting, which means that people are facing new challenges, so we’ve had a real shift towards articles that talk about how to use online modalities.

Margaret: The most noticeable change that I see is in the discussions pertaining to data collection. For instance, from 2017 to 2019 there was a greater number of submissions (that I reviewed) associated with cell phone users, e.g., investigating cell phone users who have moved and not moved and the impact on the geographic accuracy of the sample, and calculating response rate based on the treatment of working and nonworking numbers. Since 2020, the submissions (that I reviewed) leaned more towards discussions of mode, e.g., comparing messenger texting app to a traditional online survey design, design considerations for live video interviews, web-push survey delivery compared to mail-only, and phone compared to the online mode to investigate the prevalence of COVID-19.

What hasn’t changed are submissions concerning: email invitations (e.g., whether to embed the first survey question, the use of a QR code, and calling for contact information before sending the invitation); and question design (e.g., evaluating question design to understand measurement error, designing the last open-ended survey question, and whether to include a “don’t know” answer option).

5) What advice would you give to authors who are thinking about submitting their work to Survey Practice? For example, what makes a paper stand out to you as a strong submission?

Margaret: When I look back at the submissions that I have reviewed, I am reminded that the strongest submissions, in my opinion, are those that discuss: 1) an important, even fundamental, topic area that hasn’t been well investigated in the literature (e.g., the inclusion of a “don’t know” option in the design of questionnaires on sensitive topics); 2) timely/relevant topic areas (e.g., COVID); 3) new approaches (e.g., multiple methods, use of visuals); 4) compelling arguments for the usefulness of the research; and 5) strong implications for further research.

Dan: The first thing I ask myself about a submission is how applicable the problem and the solution are to other researchers. The best submissions are those that deal with a problem that a lot of other researchers are going to have, and offer up a solution that’s broadly applicable. The manuscript also needs to walk the reader through that solution, step by step.

6) On the flip side, what are the most common mistakes authors make when submitting to Survey Practice, and how can they avoid them?

Margaret: Generally speaking, I have found that the weakest areas of the submissions I have reviewed are the Methods and Discussion sections. In Methods, authors may offer shallow discussions of the research design – including details on sampling, pretesting, and mode – which leaves the reader with insufficient understanding of how the study was conducted. This is a problem because the absence of research design details creates a void of context by which to comprehend the results and ultimate usefulness of the research. In the Discussion section, authors have made the mistake of: straying from the research objectives (i.e., not grounding the discussion of results by the stated research objectives); making assumptions or predictions that are not supported by the research data; failing to discuss nuanced variations in the data that would be of interest considering the research objectives (e.g., data specific to a small population segment that may have implications related to the objectives); and missing references from the literature for support.

Dan: The most common problem I see in submissions is researchers who are too focused on proving that their solution worked. It’s important to set up the problem and make recommendations about how to deal with it, but you have to pay attention to how to put that recommendation into practice. Prove to me that what you did worked, of course, but that doesn’t help anyone if you don’t also explain how to put it into effect!

7) What advice would you give to someone considering becoming an Associate Editor for Survey Practice or another journal?

Margaret: My advice would begin with some soul searching. What I mean is that I suggest this person take the time to examine their career interests and priorities. From there, I encourage this person to consider whether their interests and priorities match the necessary commitment expected from an Associate Editor. In my case, the decision to become an Associate Editor for Survey Practice and be on the Editorial Board for Public Opinion Quarterly came easily because involvement with these journals match very well with my passion for research design and mixed methods. It is because of this passion that I welcome each new request to review, because each new review request offers me another opportunity to read what others are doing in research design and about the worthy topics they are investigating.

Beyond interests and passions, I encourage anyone who is considering an Associate Editor or board position for a journal to also evaluate practical considerations, such as time. Although I enjoy the opportunity to review submitted manuscripts, it does require a sufficient amount of devoted time from my schedule. Reviews require several readings of the manuscript and then the construction of thoughtful, useful comments to the author(s). So, I suggest taking a close look at other commitments and priorities to determine if the occasional, unpredictable review request – including the possibility of reviewing two or three revisions of any particular manuscript – is a good fit.

Dan: At any journal, the most important thing is to view the publication process as collaborative, rather than adversarial. You’re not trying to catch someone or find a reason to reject a paper; you’re trying to work with them to make the manuscript into something that’s going to fit what the journal is trying to do.

Survey Practice also requires a great deal of curiosity and flexibility. This is a big field, and there are lots of problems that I’ve just never encountered, and solutions that I never would have thought about, so it’s important to keep an open mind, and be able to go back to first principles in order to evaluate what’s being proposed in an article.