Frequentist inference is a type of statistical inference based in frequentist probability, which treats “probability” in equivalent terms to “frequency” and draws conclusions from sample-data by means of emphasizing the frequency or proportion of findings in the data. Frequentist inference underlies frequentist statistics, in which the well-established methodologies of statistical hypothesis testing and confidence intervals are founded.
The primary formulation of frequentism stems from the presumption that statistics could be perceived to have been a probabilistic frequency. This view was primarily developed by Ronald Fisher and the team of Jerzy Neyman and Egon Pearson. Ronald Fisher contributed to frequentist statistics by developing the frequentist concept of "significance testing", which is the study of the significance of a measure of a statistic when compared to the hypothesis. Neyman-Pearson extended Fisher's ideas to multiple hypotheses by conjecturing that the ratio of probabilities of hypotheses when maximizing the difference between the two hypotheses leads to a maximization of exceeding a given p-value, and also provides the basis of type I and type II errors. For more, see the foundations of statistics page.
For statistical inference, the statistic about which we want to make inferences is
y\inY
Y
\theta
\theta
\psi,λ
\psi
λ
\psi
\mu
λ
\sigma
Thus, statistical inference is concerned with the expectation of random vector
Y
E(Y)=E(Y;\theta)=\intyfY(y;\theta)dy
To construct areas of uncertainty in frequentist inference, a pivot is used which defines the area around
\psi
p
p(t,\psi)
\psi
t\inT
c
P\{p(T,\psi)\leq
| * | |
| p | |
| c\} |
P\{\psi\leqq(T,c)\}=1-c
q(t,c)
1-c
\psi
1-c
\psi
1-2c
\psi
\psi
Two complementary concepts in frequentist inference are the Fisherian reduction and the Neyman-Pearson operational criteria. Together these concepts illustrate a way of constructing frequentist intervals that define the limits for
\psi
\psi
The Fisherian reduction is defined as follows:
S
\theta
S
\psi
\psi
S=s
Essentially, the Fisherian reduction is design to find where the sufficient statistic can be used to determine the range of outcomes where
\psi
\psi
\psi
The Neyman-Pearon operational criteria is an even more specific understanding of the range of outcomes where the relevant statistic,
\psi
\psi
Because of the reliance of the Neyman-Pearson criteria on our ability to find a range of outcomes where
\psi
Frequentist inferences are associated with the application frequentist probability to experimental design and interpretation, and specifically with the view that any given experiment can be considered one of an infinite sequence of possible repetitions of the same experiment, each capable of producing statistically independent results. In this view, the frequentist inference approach to drawing conclusions from data is effectively to require that the correct conclusion should be drawn with a given (high) probability, among this notional set of repetitions.
However, exactly the same procedures can be developed under a subtly different formulation. This is one where a pre-experiment point of view is taken. It can be argued that the design of an experiment should include, before undertaking the experiment, decisions about exactly what steps will be taken to reach a conclusion from the data yet to be obtained. These steps can be specified by the scientist so that there is a high probability of reaching a correct decision where, in this case, the probability relates to a yet to occur set of random events and hence does not rely on the frequency interpretation of probability. This formulation has been discussed by Neyman, among others. This is especially pertinent because the significance of a frequentist test can vary under model selection, a violation of the likelihood principle.
Frequentism is the study of probability with the assumption that results occur with a given frequency over some period of time or with repeated sampling. As such, frequentist analysis must be formulated with consideration to the assumptions of the problem frequentism attempts to analyze. This requires looking into whether the question at hand is concerned with understanding variety of a statistic or locating the true value of a statistic. The difference between these assumptions is critical for interpreting a hypothesis test. The next paragraph elaborates on this.
There are broadly two camps of statistical inference, the epistemic approach and the epidemiological approach. The epistemic approach is the study of variability; namely, how often do we expect a statistic to deviate from some observed value. The epidemiological approach is concerned with the study of uncertainty; in this approach, the value of the statistic is fixed but our understanding of that statistic is incomplete. For concreteness, imagine trying to measure the stock market quote versus evaluating an asset's price. The stock market fluctuates so greatly that trying to find exactly where a stock price is going to be is not useful: the stock market is better understood using the epistemic approach, where we can try to quantify its fickle movements. Conversely, the price of an asset might not change that much from day to day: it is better to locate the true value of the asset rather than find a range of prices and thus the epidemiological approach is better. The difference between these approaches is non-trivial for the purposes of inference.
For the epistemic approach, we formulate the problem as if we want to attribute probability to a hypothesis. This can only be done with Bayesian statistics, where the interpretation of probability is straightforward because Bayesian statistics is conditional on the entire sample space, whereas frequentist testing is concerned with the whole experimental design. Frequentist statistics is conditioned not on solely the data but also on the experimental design. In frequentist statistics, the cutoff for understanding the frequency occurrence is derived from the family distribution used in the experiment design. For example, a binomial distribution and a negative binomial distribution can be used to analyze exactly the same data, but because their tail ends are different the frequentist analysis will realize different levels of statistical significance for the same data that assumes different probability distributions. This difference does not occur in Bayesian inference. For more, see the likelihood principle, which frequentist statistics inherently violates.[2]
For the epidemiological approach, the central idea behind frequentist statistics must be discussed. Frequentist statistics is designed so that, in the long-run, the frequency of a statistic may be understood, and in the long-run the range of the true mean of a statistic can be inferred. This leads to the Fisherian reduction and the Neyman-Pearson operational criteria, discussed above. When we define the Fisherian reduction and the Neyman-Pearson operational criteria for any statistic, we are assessing, according to these authors, the likelihood that the true value of the statistic will occur within a given range of outcomes assuming a number of repetitions of our sampling method. This allows for inference where, in the long-run, we can define that the combined results of multiple frequentist inferences to mean that a 95% confidence interval literally means the true mean lies in the confidence interval 95% of the time, but not that the mean is in a particular confidence interval with 95% certainty. This is a popular misconception.
Very commonly the epistemic view and the epidemiological view are regarded as interconvertible. This is demonstrably false. First, the epistemic view is centered around Fisherian significance tests that are designed to provide inductive evidence against the null hypothesis,
H0
Frequentist inferences stand in contrast to other types of statistical inferences, such as Bayesian inferences and fiducial inferences. While the "Bayesian inference" is sometimes held to include the approach to inferences leading to optimal decisions, a more restricted view is taken here for simplicity.
Bayesian inference is based in Bayesian probability, which treats “probability” as equivalent with “certainty”, and thus that the essential difference between the frequentist inference and the Bayesian inference is the same as the difference between the two interpretations of what a “probability” means. However, where appropriate, Bayesian inferences (meaning in this case an application of Bayes' theorem) are used by those employing frequency probability.
There are two major differences in the frequentist and Bayesian approaches to inference that are not included in the above consideration of the interpretation of probability: