Study design

Your study design specifies your research questions, hypotheses, variables, and statistical methods, enabling you to carry out your research efficiently. Designing your study lets you systematically collect and analyze your data. Moreover, it will help you write a clear paper showing your study was well thought out.

From ideas to hypotheses

Before planning your research, you must familiarize yourself with the work published in your field. For example, a study intended for several dozen participants will not be published if similar studies among thousands of patients are available. Journal editors and reviewers, who decide whether to accept a paper, judge research primarily on the originality and reliability of the results. Reporting novel results or addressing the design flaws of previous studies will thus increase your chance of being published. Conveniently, you will find the limitations of published studies in the second-to-last paragraph of the discussion section. When you get to know your research field, your initial ideas will develop into clear hypotheses, which can be translated into formal relationships between specific variables and tested with appropriate statistical methods. Only when you know your statistical methods, can you also calculate the sample size for your study.

Problems caused by a lack of study design

Collecting data without a study design is like building a house without a construction project – everything may collapse. Beginner scientists often copy paper records into a spreadsheet without specifying variables or analyses. Unfortunately, data gathered in such a way will likely be unsuitable for statistical analysis. Practice shows that data sets created without a plan typically require more time for cleaning than the intended analyses. Even when you have correct data, the lack of study design may lead to mistakes in statistical analyses. Statisticians, hoping to find publishable material, may run countless tests (fishing expeditions) or change observations to obtain significant results (p-hacking). Investigators may present only statistically significant results (cherry-picking) and formulate hypotheses after the results are known (harking). This approach violates scientific principles and makes the results difficult to understand because the analyses do not follow any pattern.

Design your study before fieldwork

It is unlikely to get publishable results without designing your study. Even when working with a complete data set, you must clarify hypotheses, choose appropriate statistical methods, and organize your results into a meaningful story. Planning your study before you start fieldwork is best because it will help you avoid mistakes uncorrectable after study completion, such as failing to measure a key variable. If you plan to ask a statistician for help, get in touch with them before you start collecting your data. The statistician will help you select an appropriate control group, avoid systematic errors (bias), and specify covariates to reduce confounding. It is always good to consult your research design with someone not engaged with your project for new insights. Be open to criticism, because it will help you improve your research and increase the chance of being published. After all, it is much better to hear criticism from a friend than from reviewers who recommend rejecting your paper.

Conclusions

The time and effort spent designing your study will always pay off. A good study design will help you:

• understand your research problem better
• address limitations uncorrectable after study completion
• select appropriate statistical methods
• organize the presentation of your results
• conduct the study according to the scientific method
• save time by not collecting unnecessary data
• write your paper efficiently

“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”

Ronald Fisher

If you plan to ask a statistician for help, get in touch with them before you start collecting your data.