What are the factors that can affect the experimental study?

Factors are the variables that experimenters control during an experiment in order to determine their effect on the response variable. A factor can take on only a small number of values, which are known as factor levels. Factors can be a categorical variable or based on a continuous variable but only use a limited number of values chosen by the experimenters.

ANOVA and design of experiments use factors extensively. For example, you are studying factors that could affect athletic performance. You decide to include the following two factors in your experiment:

Equipment brand is a categorical variable. It can only be type A or type B. On the other hand, the temperature of the room where training occurs is a continuous variable. However, in this experiment, temperature is a factor because the experimenters set only three temperatures settings: 65F, 70F and 75F.

A solution to history in this case is the randomization of experimental occasions--balanced in terms of experimenter, time of day, week and etc.

The factors described so far effect internal validity. These factors could produce changes which may be interpreted as the result of the treatment. These are called main effects which have been controlled in this design giving it internal validity.

However, in this design, there are threats to external validity (also called interaction effects because they involve the treatment and some other variable the interaction of which cause the threat to validity). It is important to note here that external validity or generalizability always turns out to involve extrapolation into a realm not represented in one's sample.

In contrast, internal validity are solvable within the limits of the logic of probability statistics. This means that we can control for internal validity based on probability statistics within the experiment conducted, however, external validity or generalizability can not logically occur because we can't logically extrapolate to different conditions. (Hume's truism that induction or generalization is never fully justified logically).

External threats include:

• Interaction of testing and X--because the interaction between taking a pretest and the treatment itself may effect the results of the experimental group, it is desirable to use a design which does not use a pretest.
• Interaction of selection and X--although selection is controlled for by randomly assigning subjects into experimental and control groups, there remains a possibility that the effects demonstrated hold true only for that population from which the experimental and control groups were selected. An example is a researcher trying to select schools to observe, however has been turned down by 9, and accepted by the 10th. The characteristics of the 10th school may be vastly different than the other 9, and therefore not representative of an average school. Therefore in any report, the researcher should describe the population studied as well as any populations which rejected the invitation.
• Reactive arrangements--this refers to the artificiality of the experimental setting and the subject's knowledge that he is participating in an experiment. This situation is unrepresentative of the school setting or any natural setting, and can seriously impact the experiment results. To remediate this problem, experiments should be incorporated as variants of the regular curricula, tests should be integrated into the normal testing routine, and treatment should be delivered by regular staff with individual students.

Research should be conducted in schools in this manner--ideas for research should originate with teachers or other school personnel. The designs for this research should be worked out with someone expert at research methodology, and the research itself carried out by those who came up with the research idea. Results should be analyzed by the expert, and then the final interpretation delivered by an intermediary.

Tests of significance for this design--although this design may be developed and conducted appropriately, statistical tests of significance are not always used appropriately.

• Wrong statistic in common use--many use a t-test by computing two ts, one for the pre-post difference in the experimental group and one for the pre-post difference of the control group. If the experimental t-test is statistically significant as opposed to the control group, the treatment is said to have an effect. However this does not take into consideration how "close" the t-test may really have been. A better procedure is to run a 2X2 ANOVA repeated measures, testing the pre-post difference as the within-subject factor, the group difference as the between-subject factor, and the interaction effect of both factors.
• Use of gain scores and covariance--the most used test is to compute pre-posttest gain scores for each group, and then to compute a t-test between the experimental and control groups on the gain scores. Also used are randomized "blocking" or "leveling" on pretest scores and the analysis of covariance are usually preferable to simple gain-score comparisons.
• Statistics for random assignment of intact classrooms to treatments--when intact classrooms have been assigned at random to treatments (as opposed to individuals being assigned to treatments), class means are used as the basic observations, and treatment effects are tested against variations in these means. A covariance analysis would use pretest means as the covariate.

The Soloman Four-Group Design

In this design, subjects are randomly assigned to four different groups: experimental with both pre-posttests, experimental with no pretest, control with pre-posttests, and control without pretests. By using experimental and control groups with and without pretests, both the main effects of testing and the interaction of testing and the treatment are controlled. Therefore generalizability increases and the effect of X is replicated in four different ways.

Statistical tests for this design--the most simple form would be the t-test. However covariance analysis and blocking on subject variables (prior grades, test scores, etc.) can be used which increase the power of the significance test similarly to what is provided by a pretest.

What are factors in an experimental study?

Which refers to the factors that affects the experiment?

What is the factor that changes and could affect the outcome of the experiment?

What are the major problems in experimental studies?