Designing Effective Surveys
Strategies for Structured Questionnaires to Measure Program Impact
Creating an effective survey requires careful consideration and strategic structuring to gather meaningful data that can inform decisions and evaluations. This article outlines best practices for structuring a survey, drawing upon insights that highlight common pitfalls and opportunities for improvement.
1. Avoid Leading Questions
Leading questions can bias responses, steering participants towards a particular answer. Questions that make assumptions about the respondent’s perspective potentially skew survey results. To avoid this, questions should be neutral, allowing participants to express their views without being influenced by the wording. For instance in the context of training, instead of asking, “How beneficial did you find the program?” consider “How would you rate the program’s impact on your learning?”
2. Break Down Complex Questions
Questions should be simple and clear. Complex or multifaceted questions can confuse respondents or force them into providing an oversimplified answer that doesn’t fully capture their experience. Breaking down these questions into more straightforward, singular-focused items can enhance clarity and elicit more accurate responses. For example, a broad question about program satisfaction could be divided into specific aspects exploring content relevance, presentation quality, and participant engagement.
3. Measure Pre and Post Program Impact
Consider how program impact will be measured. Questions relating to key program objectives should be prime candidates for a pre-post structure, allowing for the measurement of the program’s impact over time. This approach involves asking participants to reflect on their attitudes or behaviors before and after the program, providing a quantitative measure of change. For instance, querying participants about their interest in a career field before and after participation can highlight the program’s influence on career aspirations.
Get Ion Ioannidis’s stories in your inbox
Join Medium for free to get updates from this writer.
Pre and post querying can take place either literally before and after the program or at the end of the program. Pre-post querying at the end introduces bias, however this may be preferable in areas where the participants “do not know what they do not know”.
4. Guide Respondents on a Reflective Journey
The sequence of questions can significantly affect the quality of responses. A recommended practice is to structure the survey as a reflective journey, starting with more neutral or factual inquiries such as resources used, delivery methods, and activities, before moving onto questions about key outcomes and any overall feedback. Ending with the overall rating prompts respondents to recall and consider their entire experience, leading to more thoughtful evaluations.
5. Align with Program Objectives and Evaluation Frameworks
Starting with a clear understanding of the program’s logic, monitoring, evaluation framework, and key objectives before drafting any questions ensures the survey comprehensively covers all areas of interest. This alignment guarantees that the survey data can support future evaluations, funding, and decision-making processes.
6. Explore context and assumptions
Including questions about specific elements and assumptions that wrap around the program, such as prerequisite requirements for attendance, can provide valuable insights into the program’s effectiveness and areas for improvement. These questions should be carefully positioned within the survey to avoid disrupting the flow and to ensure they are relevant to all respondents.
7. Consider Privacy and Demographics
Asking demographic questions at the beginning of the survey can provide useful context for analysing responses. However, it’s crucial to balance the need for detailed information with the potential risk to privacy and the openness of feedback. Design these questions carefully to ensure they are necessary, and consider anonymizing responses to encourage honesty.
Conclusion
Structuring a survey thoughtfully can significantly enhance the quality and usefulness of the data collected. By avoiding leading questions, breaking down complex inquiries, measuring pre and post-program impact, guiding respondents through a reflective journey, aligning with program objectives, exploring context and assumption, and considering privacy and demographics, organisations can design surveys that provide actionable insights and support effective program evaluation and improvement.
