Probability of direction explained

In Bayesian statistics, the probability of direction (pd) is a measure of effect existence representing the certainty with which an effect is positive or negative.^[1] This index is numerically similar to the frequentist p-value.^{[2] [3]}

Definition

It is mathematically defined as the proportion of the posterior distribution that is of the median's sign. It typically varies between 50% and 100%.^[4]

History

The original formulation of this index and its usage in Bayesian statistics can be found in the psycho software documentation by Dominique Makowski under the appellation Maximum Probability of Effect (MPE).^{[5] [6]} It was later renamed Probability of Direction and implemented in the easystats collection of software. Similar formulations have also been described in the context of bootstrapped parameters interpretation.

Properties

The probability of direction is typically independent of the statistical model, as it is solely based on the posterior distribution and does not require any additional information from the data or the model. Contrary to indices related to the Region of Practical Interest (ROPE), it is robust to the scale of both the response variable and the predictors. However, similarly to its frequentist counterpart - the p-value, this index is not able to quantify evidence in favor of the null hypothesis.^[7] Advantages and limitations of the probability of direction have been studied by comparing it to other indices including the Bayes factor or Bayesian Equivalence test.^{[8] [9]}

Relationship with p-value

The probability of direction has a direct correspondence with the frequentist one-sided p-value through the formula

p _{one-sided} = 1 - pd and to the two-sided p-value through the formula p _{two-sided} = 2 \left( 1 - pd\right) . Thus, a two-sided p-value of respectively .1, .05, .01 and .001 would correspond approximately to a pd of 95%, 97.5%, 99.5% and 99.95%.^[10] The proximity between the pd and the p-value is in line with the interpretation of the former as an index of effect existence, as it follows the original definition of the p-value.^{[11] [12]}

Interpretation

The bayestestR package for R suggests the following rule of thumb guidelines:^[13]

! pd !! p-value equivalence !! Interpretation|-|

\leq 95 \% || p > . 1 || Uncertain|-| > 95 \% || p < . 1 || Possibly existing|-| > 97 \% || p < . 06 || Likely existing|-| > 99 \% || p < . 02 || Probably existing|-| > 99 . 9 \% || p < . 002 || Certainly existing|-|}

See also

• Bayes factor
• Equivalence test
• p-value

External links

• bayestestR - an R package for computing Bayesian indices

Notes and References

1. Makowski . Dominique . Ben-Shachar . Mattan . Lüdecke . Daniel . bayestestR: Describing Effects and their Uncertainty, Existence and Significance within the Bayesian Framework . Journal of Open Source Software . 13 August 2019 . 4 . 40 . 1541 . 10.21105/joss.01541. 2019JOSS....4.1541M . 201882316 . free .
2. Makowski . Dominique . Ben-Shachar . Mattan S. . Chen . S. H. Annabel . Lüdecke . Daniel . Indices of Effect Existence and Significance in the Bayesian Framework . Frontiers in Psychology . 10 December 2019 . 10 . 2767 . 10.3389/fpsyg.2019.02767. 31920819 . 6914840 . free .
3. Web site: Heiss . Andrew . Bayesian statistics resources . Georgia State University - Bayesian statistics course . 7 December 2021 . en-us.
4. Kelter . Riko . Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research . BMC Medical Research Methodology . December 2020 . 20 . 1 . 88 . 10.1186/s12874-020-00968-2 . 32321438 . 7178740 . free .
5. Makowski . Dominique . The psycho Package: an Efficient and Publishing-Oriented Workflow for Psychological Science . The Journal of Open Source Software . 5 February 2018 . 3 . 22 . 470 . 10.21105/joss.00470 . 2018JOSS....3..470M . free .
6. Web site: Makowski . Dominique . psycho - The Bayesian Framework . cran.r-hub.io . 26 November 2021.
7. Kelter . Riko . How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice . Multivariate Behavioral Research . 28 September 2021 . 58 . 1 . 160–188 . 10.1080/00273171.2021.1967716. 34582284 . 2005.13181 . 218900848 .
8. Baig . Sabeeh A . Bayesian Inference: Parameter Estimation for Inference in Small Samples . Nicotine & Tobacco Research . 22 October 2021 . 24 . 6 . 937–941 . 10.1093/ntr/ntab221. 34679175 . free .
9. Kelter . Riko . Bayesian and frequentist testing for differences between two groups with parametric and nonparametric two-sample tests . WIREs Computational Statistics . 2021 . 13 . 6 . e1523 . 10.1002/wics.1523 . 225532985 . free .
10. Web site: BayestestR - Probability of Direction . easystats.github.io/bayestestR . 26 November 2021 . bayestestRdocs . en.
11. Book: Fisher . R. A. . Statistical methods for research workers (11th ed. rev.) . 1925 . Oliver and Boyd . Edinburgh.
12. Cohen . Jacob . The earth is round (p < .05). . American Psychologist . 1994 . 49 . 12 . 997–1003 . 10.1037/0003-066X.49.12.997.
13. Web site: Bayesian Reporting Guidelines . easystats.github.io/bayestestR . 26 November 2021 . bayestestRguidelines . en.