Experiment & Survey Design

Research is 90% planning and 10% execution. Designing your survey or experiment correctly before data collection is perhaps the most important part of the research process. The old saying “measure twice, cut once” applies to research as well. You’ll want to make sure your procedure, measures, population, and analysis plan are all set before it’s too late. 

Concept Assistance

Our ability to help you succeed as a researcher begins with the very first step of the research project: conceptualizing your research question and the ultimate question you want to answer. We can help you sift through what’s known about your topic to create something novel and compelling around the topic or idea you’d like to own as a researcher. Once we’ve helped you identify your niche, we can help you craft the methodology for your particular project, be it a survey, an experiment, or archival research piece. We offer comprehensive assistance to graduate students and researchers – from square one and onward.

Survey Design

Most of the consultants at DataFox have backgrounds in psychology — and nobody builds a survey like a psychologist.  We are adept at making powerful surveys that can speak to your research question. Many of our clients have a good idea of what they want to study, but need help creating the survey that will create the data that speaks to their question. We’ll work with you to create a survey that fits your needs by discussing the research questions you have in mind and helping you with all aspects of the survey creation process. This includes:

• How to best order and structure your survey questions
• How to write specific survey question
• How to select the right question “anchors” 
• Determining which questions should not be asked in a survey
• Making your survey manageable by reducing survey “fatigue”
• Creating surveys that garner high amounts of engagement from participants
• Using best-practices for quality assurance during survey data collection
• Helping build your survey itself, for example, in Qualtrics, Survey Monkey, etc.

Experimental Design

Experimental design is the heart of research. Once you have a solid perspective on what you’re trying to study or show, the next step is to choose the best available experimental design for your particular project or study. This is done by considering your specific hypothesis and population of interest. We design powerful experiments by soundly establishing validity – making sure your study accurately captures what it’s intended to capture. This is done by building experiments that are valid at their core. We do this by: 

• Understanding the nuances of construct validity, that is, ensuring the questions asked are actually capturing the desired constructs, and not superfluous or seemingly related other ones. 

• Sampling in ways that maximize external validity. Often times, researchers have particular demographics or populations they want their studies to generalize to. We help researchers ensure their sampling methods and survey distribution generalize so they can make valid real-world, high-impact claims about their results. 
• Creating powerful questions. We are experts at understanding which scales are most appropriate to use for particular questions, and how using different item responses can leverage small effects into tangible differences. 
• Designing strong manipulations. Creating a study capable of achieving significant results will depend on the ability of your manipulation (or treatment) to produce a detectable shift in some kind of outcome variable. We help create powerful studies by designing manipulations that are potent while controlling and eliminating extraneous “noise” that dilutes desired effects. 
• Creating strong internal validity– that is, creating the experiment in a way such that we can pin down exactly what causes the outcome, not just what correlates with it.
• Testing for criterion-related validity – the extent to which we can not just accurately model, but then use the models to make real-world predictions.