What is the importance of determining the appropriate sample size for a research study?
|
Few of us read research reports with an eye to critiquing the methodology. The results are the main attraction, the reason for reading in the first place. But researchers spend much of their time planning how their studies will be carried out. Shouldn’t we pay more attention? As any decent researcher will tell you, a study’s results are only as good as its design. Sample size and power are key elements of study design. Show
What is sample size and why is it important?Sample size refers to the number of participants or observations included in a study. This number is usually represented by n. The size of a sample influences two statistical properties: 1) the precision of our estimates and 2) the power of the study to draw conclusions. To use an example, we might choose to compare the performance of marathon runners who eat oatmeal for breakfast to the performance of those who do not. Since it would be impossible to track the dietary habits of every marathon runner in the world, we have little choice but to focus on a segment of that larger population. This might mean randomly selecting only 100 runners for our study. The sample size, or n, in this scenario is 100. The study’s findings could describe the population of all runners based on the information obtained from the sample of 100 runners. No matter how careful we are about choosing our 100 runners, there will still be some margin of error in the study results. This is because we haven’t talked to everyone in our population of interest. We can’t be absolutely precise about how eating oatmeal affects running performance because it would be impossible to look at every instance in which these two activities coincide. This measure of error is known as sampling error. It influences the precision of our description of the population of all runners. Sampling error, though unavoidable, can be eased by sample size. Larger samples tend to be associated with a smaller margin of error. This makes sense. To get an accurate picture of the effects of eating oatmeal on running performance, we need plenty of examples to look at and compare. However, there is a point at which increasing sample size no longer impacts the sampling error. This phenomenon is known as the law of diminishing returns. What about power?Clearly, determining the right sample size is crucial for strong experimental design. But what about power? Power refers to the probability of finding a statistically significant result (read the column on statistical significance). In our study of marathon runners, power is the probability of finding a difference in running performance that is related to eating oatmeal. We calculate power by specifying two alternative scenarios. The first, called the null hypothesis, is one that says there’s nothing going on in the population of interest. In our study of marathoners, the null hypothesis might say that eating oatmeal has no effect on performance. The second is the alternative hypothesis. This is the often anticipated outcome of the study. In our example, it might be that eating oatmeal results in consistently better performance. The power equation uses these two alternatives so that the study can find the answer to the research question. As researchers, we want to know if our study of marathoners can detect the difference between oatmeal having no impact on running performance (the null hypothesis) and oatmeal having a considerable impact on running performance (the alternative hypothesis). Often researchers will begin a study by asking what sample size is necessary to produce a desirable power. This process is known as a priori power analysis. It shows nicely how sample size and power are inter-related. A larger sample size gives more power. While the particulars of calculating sample size and power are best left to the experts, even the most mathematically-challenged of us can benefit from understanding a little bit about study design. The next time you read a research report, take a look at the methodology. You never know. It just might change the way you read the results. Source: At Work, Issue 53, Summer 2008: Institute for Work & Health, Toronto When conducting research, quality sampling may be characterized by the number and selection of subjects or observations. Obtaining a sample size that is appropriate in both regards is critical for many reasons. Most importantly, a large sample size is more representative of the population, limiting the influence of outliers or extreme observations. A sufficiently large sample size is also necessary to produce results among variables that are significantly different.(1) For qualitative studies, where the goal is to “reduce the chances of discovery failure,” a large sample size broadens the range of possible data and forms a better picture for analysis.(2) Sample size is also important for economic and ethical reasons. As Russell Lenth from the University of Iowa explains, “An under-sized study can be a waste of resources for not having the capability to produce useful results, while an over-sized one uses more resources than are necessary. In an experiment involving human or animal subjects, sample size is a pivotal issue for ethical reasons. An under-sized experiment exposes the subjects to potentially harmful treatments without advancing knowledge. In an over-sized experiment, an unnecessary number of subjects are exposed to a potentially harmful treatment, or are denied a potentially beneficial one.”(3) Theoretical Case Study: Dangers of Small Sample SizeIn an article on sample size in qualitative research, a marketing research consultant gives the example of a study conducted on patient satisfaction in a medical clinic. The medical clinic has one staff member known to aggravate 1 out of every 10 patients visiting. A research budget permits only one focus group with 10 clinic patients, and all respondents report feeling satisfied with their visit. However, when performing data analysis, it is critical to consider the population represented by a study of only ten patients. The probability that the sample failed to include an unsatisfied patient is calculated to be 35%. In other words, approximately 1 in 3 random samples of ten patients would overlook the actual statistic of aggravation (1 out of every 10 patients). To see how this calculation was performed, visit: http://www.icology.co.uk/qualitativesamplesize.html.(4) Determining Sample Size:There are many different ways to determine an appropriate sample size. For in-depth qualitative studies, Abbie Griffin and John Hauser found that “20-30 in-depth interviews are necessary to uncover 90-95% of all customer needs for the product categories studied.”(5) Thus, the authors determined that a sample size of 30 respondents would provide a reasonable starting point. This number is corroborated by Dr. Saiful, a clinical researcher, who states that a “sample size larger than 30 and less than 500 are appropriate for most research,” adding that sub-samples also require at least 30 observations when applicable.(6) Determining the exact sample size necessary for a study usually requires extensive statistical calculations. However, a reasonable sample size acceptable in most studies utilizes the calculated margin of error. An estimation of margin of error at 95% confidence level (where there is only a 5% chance that the sample results differ from the true population) is given by 1/√N, where N is the number of participants or sample size. This means that a sample size of 10 would have a 31.6% margin of error (1/√10=0.316). To demonstrate this calculation through example, we can walk-through a study on fear of heights. If researchers survey 10 people and find that 6 respondents are afraid of heights, this means that there is a 95% chance that between 2.8 (6 – 3.16) and 9.2 (6 + 3.16) of the population is actually afraid of heights. With such a large range, the data is not very conclusive. However, if the researchers survey 100 people, the margin of error falls to 10%. Now, if 60 participants report a fear of heights, there is a 95% chance that between 50 (60 – 10) and 70 (60 + 10) of the population actually has a fear of heights. The greater N is, the smaller the margin of error and more useful the measurable results.(7) In addition to the yield of statistical significance and confidence in results, quality sample size must consider the rate of response. Incomplete or illegible responses are not useful observations. Thus, the total sample size must account for these potential issues.(8) Methods of Sampling:Purposive SamplingA common strategy for sampling in qualitative research studies, purposive sampling places participants in groups relevant to criteria that fits the research question. Factors that affect sample size include available resources, study time, and objectives. However, sample sizes are also determined by the concept of “theoretical saturation,” or “the point in data collection when new data no longer bring additional insights to the research questions.”(9) Generally, studies that use purposive sampling have a target number of participants, rather than a set requirement. Quota SamplingQuota sampling predetermines the number of participants desired. While designing the study, researchers may determine sample size, along with appropriate proportions of subsamples, when identifying participants of certain characteristics. With this criteria, researchers can then recruit participants appropriate to the “location, culture, and study population…until [meeting] the prescribed quotas.”(10) Snowball SamplingThis third type of sampling uses existing participants or contacts to reach their social networks and refer the researcher to other potential participants. Snowball sampling helps to recruit “hidden populations” that may not be found from other methods of sampling.(11) Strategies to Obtain a Quality Sample:
Case Study: Participant Recruitment in Reproductive Health Research in IndiaA study conducted in India on reproductive health found that when female recruiters approached patients in the waiting room of an outpatient OB/GYN clinic, only 23% of those screened were eligible for the study. Overall, only 7% of those screened enrolled. When recruiters adopted an alternative method to utilize community resources and networks to find participants, they found greater success. By inquiring within women’s microeconomic self-help groups, 87.9% of those screened were eligible for the study. Of the women screened, 85.2% enrolled in the study. Moreover, those recruited in community clinics had higher retention rates and were more likely to attend their first follow-up visit. In particular, 97% of recruitments from community groups attended their first-follow up, while only 72% of participants recruited from the clinic attended. The drastic differences in enrollment and retention between the two methods suggests that a “community-supported recruitment process may facilitate access to young women in the community, increase general knowledge and health seeking on reproductive health issues, and produced better overall study retention.”(14) The study suggests reasons for low recruitment through clinics as patient fear of healthcare settings, barriers to transportation, social stigma from attending the clinic, and restricted female autonomy. While sociobehavioral research may use findings to explore such issues, this case study demonstrates the value of sampling strategies, including the employment of community infrastructure and the need for flexibility throughout the sampling process.(15) Footnotes(1) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003) (2) DePaulo, P. (2000). Sample size for qualitative research. Quirks Marketing Research Review, 1202. (3) Lenth, R. V. (2001). Some practical guidelines for effective sample size determination. The American Statistician, 55(3), 187-193. (4) DePaulo, P. (2000). Sample size for qualitative research. Quirks Marketing Research Review, 1202. (8) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003). (13) Patel, M., Doku, V., and Tennakoon, L. “Challenges in recruitment of research participants.”Advances in Psychiatric Treatment. 9. (2003). (14) Krupp, K., et. al. “Novel recruitment strategies to increase participation of women in reproductive health research in India.”Global Public Health. 2.4 (2007). What is the importance of appropriate determination of sample size?The sample size for a study needs to be estimated at the time the study is proposed; too large a sample is unnecessary and unethical, and too small a sample is unscientific and also unethical. The necessary sample size can be calculated, using statistical software, based on certain assumptions.
Why is it important that researchers determine appropriate sample size with consideration to the level of significance effect size and power?Determining the optimal sample size for a study assures an adequate power to detect statistical significance. Hence, it is a critical step in the design of a planned research protocol. Using too many participants in a study is expensive and exposes more number of subjects to procedure.
Why is determining the sample important in research?Sampling saves money by allowing researchers to gather the same answers from a sample that they would receive from the population. Non-random sampling is significantly cheaper than random sampling, because it lowers the cost associated with finding people and collecting data from them.
What is the importance of determining the sample size and choosing the appropriate sampling technique in conducting a study?The Importance of Selecting an Appropriate Sampling Method
Sampling yields significant research result. However, with the differences that can be present between a population and a sample, sample errors can occur. Therefore, it is essential to use the most relevant and useful sampling method.
|
