Ordinal priority approach explained

Ordinal priority approach (OPA) is a multiple-criteria decision analysis method that aids in solving the group decision-making problems based on preference relations.

Description

Various methods have been proposed to solve multi-criteria decision-making problems.[1] The basis of methods such as analytic hierarchy process and analytic network process is pairwise comparison matrix.[2] The advantages and disadvantages of the pairwise comparison matrix were discussed by Munier and Hontoria in their book.[3] In recent years, the OPA method was proposed to solve the multi-criteria decision-making problems based on the ordinal data instead of using the pairwise comparison matrix.[4] The OPA method is a major part of Dr. Amin Mahmoudi's PhD thesis from the Southeast University of China.This method uses linear programming approach to compute the weights of experts, criteria, and alternatives simultaneously.[5] The main reason for using ordinal data in the OPA method is the accessibility and accuracy of the ordinal data compared with exact ratios used in group decision-making problems involved with humans.[6]

In real-world situations, the experts might not have enough knowledge regarding one alternative or criterion. In this case, the input data of the problem is incomplete, which needs to be incorporated into the linear programming of the OPA. To handle the incomplete input data in the OPA method, the constraints related to the criteria or alternatives should be removed from the OPA linear-programming model.[7]

Various types of data normalization methods have been employed in multi-criteria decision-making methods in recent years. Palczewski and Sałabun showed that using various data normalization methods can change the final ranks of the multi-criteria decision-making methods.[8] Javed and colleagues showed that a multiple-criteria decision-making problem can be solved by avoiding the data normalization.[9] There is no need to normalize the preference relations and thus, the OPA method does not require data normalization.

The OPA method

The OPA model is a linear programming model, which can be solved using a simplex algorithm. The steps of this method are as follows: Step 1: Identifying the experts and determining the preference of experts based on their working experience, educational qualification, etc. Step 2: identifying the criteria and determining the preference of the criteria by each expert. Step 3: identifying the alternatives and determining the preference of the alternatives in each criterion by each expert. Step 4: Constructing the following linear programming model and solving it by an appropriate optimization software such as LINGO, GAMS, MATLAB, etc.

\begin&Max Z \\&S.t. \\&Z \leq r_\bigg (r_ \big(r_ (w_^ - w_^) \big)\bigg) \; \; \; \; \forall i,j \; and \; r_ \\&Z \leq r_ r_ r_ w_^ \; \; \; \forall i,j \; and \; r_ \\&\sum_^\sum_^\sum_^ w_ = 1 \\&w_\geq0 \; \; \; \forall i, j \; and \; k \\&Z: Unrestricted\;in\;sign \\\end

In the above model,

ri(i=1,...,p)

represents the rank of expert

i

,

rj(j=1...,n)

represents the rank of criterion

j

,

rk(k=1...,m)

represents the rank of alternative

k

, and

wijk

represents the weight of alternative

k

in criterion

j

by expert

i

.After solving the OPA linear programming model, the weight of each alternative is calculated by the following equation:

\begin{aligned} &wk=\sum

p
i=1
n
\sum
j=1

wijk\forallk\\ \end{aligned}

The weight of each criterion is calculated by the following equation:

\begin{aligned} &wj=\sum

p
i=1
m
\sum
k=1

wijk\forallj\\ \end{aligned}

And the weight of each expert is calculated by the following equation:

\begin{aligned} &wi=\sum

n
j=1
m
\sum
k=1

wijk\foralli\\ \end{aligned}

Example

Suppose that we are going to investigate the issue of buying a house. There are two experts in this decision problem. Also, there are two criteria called cost (c), and construction quality (q) for buying the house. On the other hand, there are three houses (h1, h2, h3) for purchasing. The first expert (x) has three years of working experience and the second expert (y) has two years of working experience. The structure of the problem is shown in the figure.

Step 1: The first expert (x) has more experience than expert (y), hence x > y.

Step 2: The criteria and their preference are summarized in the following table:

Experts’ opinions regarding criteria!Criteria!Expert (x)!Expert (y)
c12
q21

Step 3: The alternatives and their preference are summarized in the following table:

Alternatives! colspan="2"
Expert (x)Expert (y)
cqcq
h11213
h23121
h32332

Step 4: The OPA linear programming model is formed based on the input data as follows:

\begin{align} &MaxZ\\ &S.t.\\ &Z\leq1*1*1*(wxch1-wxch3)\\ &Z\leq1*1*2*(wxch3-wxch2)\\ &Z\leq1*1*3*wxch2\\ \\ &Z\leq1*2*1*(wxqh2-wxqh1)\\ &Z\leq1*2*2*(wxqh1-wxqh3)\\ &Z\leq1*2*3*wxqh3\\ \\ &Z\leq2*2*1*(wych1-wych2)\\ &Z\leq2*2*2*(wych2-wych3)\\ &Z\leq2*2*3*wych3\\ \\ &Z\leq2*1*1*(wyqh2-wyqh3)\\ &Z\leq2*1*2*(wyqh3-wyqh1)\\ &Z\leq2*1*3*wyqh1\\ \\ &wxch1+wxch2+wxch3+wxqh1+wxqh2+wxqh3+wych1+wych2+wych3+wyqh1+wyqh2+wyqh3=1\\ \\ \end{align}

After solving the above model using optimization software, the weights of experts, criteria and alternatives are obtained as follows:

\begin{align}&wx=wxch1+wxch2+wxch3+wxqh1+wxqh2+wxqh3=0.666667\\\\&wy=wych1+wych2+wych3+wyqh1+wyqh2+wyqh3=0.333333\\\\\\&wc=wxch1+wxch2+wxch3+wych1+wych2+wych3=0.555556\\\\&wq=wxqh1+wxqh2+wxqh3+wyqh1+wyqh2+wyqh3=0.444444\\\\\\&wh1=wxch1+wxqh1+wych1+wyqh1=0.425926\\\\&wh2=wxch2+wxqh2+wych2+wyqh2=0.351852\\\\&wh3=wxch3+wxqh3+wych3+wyqh3=0.222222\\\\\end{align}

Therefore, House 1 (h1) is considered as the best alternative. Moreover, we can understand that criterion cost (c) is more important than criterion construction quality (q). Also, based on the experts' weights, we can understand that expert (x) has a higher impact on final selection compared with expert (y).

Applications

The applications of the OPA method in various field of studies are summarized as follows:

Agriculture, manufacturing, services

Construction industry

Energy and environment

Healthcare

Information technology

Transportation

Extensions

Several extensions of the OPA method are listed as follows:

Software

The following non-profit tools are available to solve the MCDM problems using the OPA method:

Notes and References

  1. Mardani . Abbas . Jusoh . Ahmad . MD Nor . Khalil . Khalifah . Zainab . Zakwan . Norhayati . Valipour . Alireza . 2015 . Multiple criteria decision-making techniques and their applications – a review of the literature from 2000 to 2014 . Economic Research-Ekonomska Istraživanja . 28 . 1 . 516–571 . 10.1080/1331677x.2015.1075139 . 57402259 . 1331-677X . 2022-09-23 . 2022-09-23 . https://web.archive.org/web/20220923134425/https://www.tandfonline.com/doi/full/10.1080/1331677X.2015.1075139 . live . free .
  2. Penadés-Plà . Vicent . García-Segura . Tatiana . Martí . José . Yepes . Víctor . 2016-12-09 . A Review of Multi-Criteria Decision-Making Methods Applied to the Sustainable Bridge Design . Sustainability . 8 . 12 . 1295 . 10.3390/su8121295 . 2071-1050 . free .
  3. Book: Munier . Nolberto . Uses and Limitations of the AHP Method . Hontoria . Eloy . Management for Professionals . 2021 . Springer Nature . 10.1007/978-3-030-60392-2 . 978-3-030-60392-2 . 241759250 . en . 2022-09-19 . 2022-09-23 . https://web.archive.org/web/20220923134500/https://link.springer.com/book/10.1007/978-3-030-60392-2 . live .
  4. Ataei . Younes . Mahmoudi . Amin . Feylizadeh . Mohammad Reza . Li . Deng-Feng . 1 January 2020 . Ordinal Priority Approach (OPA) in Multiple Attribute Decision-Making . Applied Soft Computing . 86 . 105893 . 10.1016/j.asoc.2019.105893 . 209928171.
  5. Sotoudeh-Anvari . Alireza . 1 September 2022 . The applications of MCDM methods in COVID-19 pandemic: A state of the art review . Applied Soft Computing . 126 . 109238 . 10.1016/j.asoc.2022.109238. 35795407 . 9245376 .
  6. Wang . Haomin . Peng . Yi . Kou . Gang . 1 July 2021 . A two-stage ranking method to minimize ordinal violation for pairwise comparisons . Applied Soft Computing . 106 . 107287 . 10.1016/j.asoc.2021.107287. 233657592 . free .
  7. Mahmoudi . Amin . Deng . Xiaopeng . Javed . Saad Ahmed . Yuan . Jingfeng . 2021-10-01 . Large-scale multiple criteria decision-making with missing values: project selection through TOPSIS-OPA . live . Journal of Ambient Intelligence and Humanized Computing . en . 12 . 10 . 9341–9362 . 10.1007/s12652-020-02649-w . 1868-5145 . https://web.archive.org/web/20220923134422/https://link.springer.com/article/10.1007/s12652-020-02649-w . 2022-09-23 . 2022-09-23 . 228929310.
  8. Palczewski . Krzysztof . Sałabun . Wojciech . 2019 . Influence of various normalization methods in PROMETHEE II: an empirical study on the selection of the airport location . Procedia Computer Science . 159 . 2051–2060 . 10.1016/j.procs.2019.09.378 . 207756779 . 1877-0509 . 2022-09-22 . 2022-09-23 . https://web.archive.org/web/20220923134405/https://www.sciencedirect.com/science/article/pii/S1877050919315819?via%3Dihub . live . free .
  9. Ahmed Javed . Saad . Gunasekaran . Angappa . Mahmoudi . Amin . 2022-09-22 . DGRA: Multi-sourcing and Supplier Classification through Dynamic Grey Relational Analysis Method . Computers & Industrial Engineering . 173 . en . 108674 . 10.1016/j.cie.2022.108674 . 252478074 . 0360-8352.
  10. Mahmoudi . Amin . Sadeghi . Mahsa . Naeni . Leila Moslemi . 2023 . Blockchain and supply chain finance for sustainable construction industry: ensemble ranking using Ordinal Priority Approach . Operations Management Research . en . 10.1007/s12063-023-00374-z . 259410408 . 1936-9743.
  11. Ahmed Javed . Saad . Gunasekaran . Angappa . Mahmoudi . Amin . 2022-09-22 . DGRA: Multi-sourcing and Supplier Classification through Dynamic Grey Relational Analysis Method . Computers & Industrial Engineering . 173 . en . 108674 . 10.1016/j.cie.2022.108674 . 0360-8352 . 252478074.
  12. Le . Minh-Tai . Nhieu . Nhat-Luong . January 2022 . A Novel Multi-Criteria Assessment Approach for Post-COVID-19 Production Strategies in Vietnam Manufacturing Industry: OPA - Fuzzy EDAS Model . Sustainability . 14 . 8 . 4732 . 10.3390/su14084732 . free.
  13. Tafakkori . Keivan . Tavakkoli-Moghaddam . Reza . Siadat . Ali . 2022 . Sustainable negotiation-based nesting and scheduling in additive manufacturing systems: A case study and multi-objective meta-heuristic algorithms . live . Engineering Applications of Artificial Intelligence . 112 . 104836 . 10.1016/j.engappai.2022.104836 . 0952-1976 . https://web.archive.org/web/20220923134408/https://www.sciencedirect.com/science/article/abs/pii/S0952197622000975?via%3Dihub . 2022-09-23 . 2022-09-20 . 247829389. free .
  14. Bah . M. K. . Tulkinov . S. . 2022-07-20 . Evaluation of Automotive Parts Suppliers through Ordinal Priority Approach and TOPSIS Management Science and Business Decisions . live . en-US . 10.52812/msbd.37 . https://web.archive.org/web/20220721064058/https://publish.thescienceinsight.com/index.php/msbd/article/view/37 . 2022-07-21 . 2022-09-19 . 250934141. free .
  15. Li . Jintao . Dai . Yan . Wang . Cynthia Changxin . Sun . Jun . 2022 . Assessment of Environmental Demands of Age-Friendly Communities from Perspectives of Different Residential Groups: A Case of Wuhan, China . International Journal of Environmental Research and Public Health . en . 19 . 15 . 9120 . 10.3390/ijerph19159120 . 1660-4601 . 9368052 . 35897508 . free.
  16. Mahmoudi . Amin . Javed . Saad Ahmed . April 2022 . Performance Evaluation of Construction Sub‐contractors using Ordinal Priority Approach . Evaluation and Program Planning . 91 . 102022 . 10.1016/j.evalprogplan.2021.102022 . 34736766 . 239609916.
  17. Mahmoudi . Amin . Sadeghi . Mahsa . Deng . Xiaopeng . 2022-04-12 . Performance measurement of construction suppliers under localization, agility, and digitalization criteria: Fuzzy Ordinal Priority Approach . Environment, Development and Sustainability . 1–26 . 10.1007/s10668-022-02301-x . 1387-585X . 9001166 . 35431618.
  18. Faisal . Mohd. Nishat . Al Subaie . Abdulla Abdulaziz . Sabir . Lamay Bin . Sharif . Khurram Jahangir . 2022-01-01 . PMBOK, IPMA and fuzzy-AHP based novel framework for leadership competencies development in megaprojects . live . Benchmarking. 10.1108/BIJ-10-2021-0583 . 1463-5771 . https://web.archive.org/web/20220923134423/https://www.emerald.com/insight/content/doi/10.1108/BIJ-10-2021-0583/full/html . 2022-09-23 . 2022-09-19 . 250940618.
  19. Quayson . Matthew . Bai . Chunguang . Mahmoudi . Amin . Hu . Weihao . Chen . Wei . Omoruyi . Osayuwamen . 2023 . Designing a decision support tool for integrating ESG into the natural resource extraction industry for sustainable development using the ordinal priority approach . Resources Policy . 85 . 103988 . 10.1016/j.resourpol.2023.103988 . 260727664 . 0301-4207.
  20. Ala . Ali . Mahmoudi . Amin . Mirjalili . Seyedali . Simic . Vladimir . Pamucar . Dragan . 2023 . Evaluating the Performance of various Algorithms for Wind Energy Optimization: A Hybrid Decision-Making model . Expert Systems with Applications . 221 . 119731 . 10.1016/j.eswa.2023.119731 . 257212701 . 0957-4174.
  21. Elkadeem . Mohamed R. . Younes . Ali . Mazzeo . Domenico . Jurasz . Jakub . Elia Campana . Pietro . Sharshir . Swellam W. . Alaam . Mohamed A. . 2022 . Geospatial-assisted multi-criterion analysis of solar and wind power geographical-technical-economic potential assessment . Applied Energy . 322 . 119532 . 10.1016/j.apenergy.2022.119532 . 0306-2619 . 250062623.
  22. Islam . Shajedul . 2021-07-28 . Evaluation of Low-Carbon Sustainable Technologies in Agriculture Sector through Grey Ordinal Priority Approach . live . International Journal of Grey Systems . 1 . 1 . 5–26 . 10.52812/ijgs.3 . 2767-3308 . https://web.archive.org/web/20220923134407/http://publish.thescienceinsight.com/index.php/ijgs/article/view/3 . 2022-09-23 . 2022-09-19 . 237463151. free .
  23. Candra . Cliford Septian . 2022-07-29 . Evaluation of Barriers to Electric Vehicle Adoption in Indonesia through Grey Ordinal Priority Approach International Journal of Grey Systems . en-US . 10.52812/ijgs.46. 251183598 . free .
  24. Quartey-Papafio . T. K. . Shajedul . I. . Dehaghani . A. R. . 2021-07-25 . Evaluating Suppliers for Healthcare Centre using Ordinal Priority Approach Management Science and Business Decisions . live . en-US . 10.52812/msbd.12 . https://web.archive.org/web/20210804002122/http://publish.thescienceinsight.com/index.php/msbd/article/view/12 . 2021-08-04 . 2022-09-19 . 237950190. free .
  25. Dorado Chaparro . Javier . Fernández-Bermejo Ruiz . Jesús . Santofimia Romero . María José . del Toro García . Xavier . Cantarero Navarro . Rubén . Bolaños Peño . Cristina . Llumiguano Solano . Henry . Villanueva Molina . Félix Jesús . Gonçalves Silva . Anabela . López . Juan Carlos . 2022-05-01 . Phyx.io: Expert-Based Decision Making for the Selection of At-Home Rehabilitation Solutions for Active and Healthy Aging . International Journal of Environmental Research and Public Health . en . 19 . 9 . 5490 . 10.3390/ijerph19095490 . 1660-4601 . 9103419 . 35564884 . free.
  26. Pamucar . Dragan . Deveci . Muhammet . Gokasar . Ilgin . Tavana . Madjid . Köppen . Mario . 2022 . A metaverse assessment model for sustainable transportation using ordinal priority approach and Aczel-Alsina norms . live . Technological Forecasting and Social Change . 182 . 121778 . 10.1016/j.techfore.2022.121778 . 0040-1625 . https://web.archive.org/web/20220923134405/https://www.sciencedirect.com/science/article/abs/pii/S004016252200302X?via%3Dihub . 2022-09-23 . 2022-09-19 . 249799590. 10044/1/98484 . free .
  27. Deveci . Muhammet . Pamucar . Dragan . Gokasar . Ilgin . Koppen . Mario . Gupta . Brij B. . 2022 . Personal Mobility in Metaverse With Autonomous Vehicles Using Q-Rung Orthopair Fuzzy Sets Based OPA-RAFSI Model . IEEE Transactions on Intelligent Transportation Systems . 1–10 . 10.1109/TITS.2022.3186294 . 250507795.
  28. Deveci . Muhammet . Pamucar . Dragan . Gokasar . Ilgin . Pedrycz . Witold . Wen . Xin . 2022 . Autonomous Bus Operation Alternatives in Urban Areas Using Fuzzy Dombi-Bonferroni Operator Based Decision Making Model . live . IEEE Transactions on Intelligent Transportation Systems . 1–10 . 10.1109/TITS.2022.3202111 . 1524-9050 . https://web.archive.org/web/20220923134404/https://ieeexplore.ieee.org/document/9894368/ . 2022-09-23 . 2022-09-19 . 252349294.
  29. Su . Chong . Ma . Xuri . Lv . Jing . Tu . Tao . Li . Hongguang . 2022 . A multilayer affective computing model with evolutionary strategies reflecting decision-makers' preferences in process control . live . ISA Transactions . 128 . Pt B . 565–578 . 10.1016/j.isatra.2021.11.038 . 0019-0578 . 34953588 . https://web.archive.org/web/20220923134408/https://www.sciencedirect.com/science/article/abs/pii/S0019057821006133?via%3Dihub . 2022-09-23 . 2022-09-19 . 245168890.
  30. Sadeghi . Mahsa . Mahmoudi . Amin . Deng . Xiaopeng . February 2022 . Adopting distributed ledger technology for the sustainable construction industry: evaluating the barriers using Ordinal Priority Approach . Environmental Science and Pollution Research . 29 . 7 . 10495–10520 . 10.1007/s11356-021-16376-y . 8443118 . 34528198.
  31. Sadeghi . Mahsa . Mahmoudi . Amin . Deng . Xiaopeng . 19 April 2022 . Blockchain technology in construction organizations: risk assessment using trapezoidal fuzzy ordinal priority approach . Engineering, Construction and Architectural Management . ahead-of-print . ahead-of-print . 10.1108/ECAM-01-2022-0014 . 248225580.
  32. Sadeghi . M. . Mahmoudi . A. . Deng . X. . Luo . X. . 27 June 2022 . Prioritizing requirements for implementing blockchain technology in construction supply chain based on circular economy: Fuzzy Ordinal Priority Approach . International Journal of Environmental Science and Technology . 20 . 5 . 4991–5012 . 10.1007/s13762-022-04298-2 . 250065647.
  33. Amirghodsi . Sirous . Naeini . Ali Bonyadi . Makui . Ahmad . 2022 . An Integrated Delphi-DEMATEL-ELECTRE Method on Gray Numbers to Rank Technology Providers . live . IEEE Transactions on Engineering Management . 69 . 4 . 1348–1364 . 10.1109/tem.2020.2980127 . 0018-9391 . https://web.archive.org/web/20220923134406/https://ieeexplore.ieee.org/document/9083974/ . 2022-09-23 . 2022-09-19 . 218924240.
  34. Pamucar . Dragan . Deveci . Muhammet . Gokasar . Ilgin . Martínez . Luis . Köppen . Mario . 1 July 2022 . Prioritizing transport planning strategies for freight companies towards zero carbon emission using ordinal priority approach . Computers & Industrial Engineering . 169 . 108259 . 10.1016/j.cie.2022.108259 . 248978509. free . 10044/1/98486 . free .
  35. Bouraima . Mouhamed Bayane . Kiptum . Clement Kiprotich . Ndiema . Kevin Maraka . Qiu . Yanjun . Tanackov . Ilija . 2022-08-19 . Prioritization Road Safety Strategies Towards Zero Road Traffic Injury Using Ordinal Priority Approach . live . Operational Research in Engineering Sciences: Theory and Applications . en . 5 . 2 . 206–221 . 10.31181/oresta190822150b . 2620-1747 . https://web.archive.org/web/20220821152434/https://oresta.rabek.org/index.php/oresta/article/view/291 . 2022-08-21 . 2022-09-19 . 251728499. free .
  36. Bouraima . Mouhamed Bayane . Qiu . Yanjun . Kiptum . Clement Kiprotich . Ndiema . Kevin Maraka . 2022-08-17 . Evaluation of Factors Affecting Road Maintenance in Kenyan Counties Using the Ordinal Priority Approach . live . Journal of Computational and Cognitive Engineering . en . 10.47852/bonviewJCCE2202272 . 2810-9503 . https://web.archive.org/web/20220923134408/https://ojs.bonviewpress.com/index.php/JCCE/article/view/272 . 2022-09-23 . 2022-09-19. free .
  37. Mahmoudi . Amin . Deng . Xiaopeng . Javed . Saad Ahmed . Zhang . Na . January 2021 . Sustainable Supplier Selection in Megaprojects: Grey Ordinal Priority Approach . Business Strategy and the Environment . 30 . 1 . 318–339 . 10.1002/bse.2623 . 224917346.
  38. Mahmoudi . Amin . Javed . Saad Ahmed . Mardani . Abbas . June 2022 . Gresilient supplier selection through Fuzzy Ordinal Priority Approach: decision-making in post-COVID era . Operations Management Research . 15 . 1–2 . 208–232 . 10.1007/s12063-021-00178-z . 232240914. free .
  39. Mahmoudi . Amin . Javed . Saad Ahmed . 2023 . Uncertainty Analysis in Group Decisions through Interval Ordinal Priority Approach . Group Decision and Negotiation . 32 . 4 . 807–833 . 10.1007/s10726-023-09825-1 . 258205868 . 0926-2644.
  40. Mahmoudi . Amin . Javed . Saad Ahmed . 2023 . Strict and weak ordinal relations for estimating the criteria weights in Ordinal Priority Approach (OPA) . MethodsX . en . 11 . 102389 . 10.1016/j.mex.2023.102389. 10542415 .
  41. Mahmoudi . Amin . Javed . Saad Ahmed . October 2022 . Probabilistic Approach to Multi-Stage Supplier Evaluation: Confidence Level Measurement in Ordinal Priority Approach . Group Decision and Negotiation . 31 . 5 . 1051–1096 . 10.1007/s10726-022-09790-1 . 9409630 . 36042813.
  42. Abdel-Basset . Mohamed . Mohamed . Mai . Abdel-monem . Ahmed . Elfattah . Mohamed Abd . 2022-04-29 . New extension of ordinal priority approach for multiple attribute decision-making problems: design and analysis . live . Complex & Intelligent Systems . 8 . 6 . 4955–4970 . 10.1007/s40747-022-00721-w . 2199-4536 . 9051802 . 35505994 . https://web.archive.org/web/20220923134405/https://link.springer.com/article/10.1007/s40747-022-00721-w . 2022-09-23 . 2022-09-23.
  43. Du . Jun-Liang . Liu . Si-Feng . Javed . Saad Ahmed . Goh . Mark . Chen . Zhen-Song . 2023 . Enhancing Quality Function Deployment Through the Integration of Rough Set and Ordinal Priority Approach: A Case Study in Electric Vehicle Manufacturing . IEEE Transactions on Engineering Management . 1–12 . 10.1109/TEM.2023.3282228 . 1558-0040.
  44. Mahmoudi . Amin . Abbasi . Mehdi . Deng . Xiaopeng . February 2022 . A novel project portfolio selection framework towards organizational resilience: Robust Ordinal Priority Approach . Expert Systems with Applications . 188 . 116067 . 10.1016/j.eswa.2021.116067. 36818824 . 9928571 .
  45. Mahmoudi . Amin . Abbasi . Mehdi . Deng . Xiaopeng . April 2022 . Evaluating the Performance of the Suppliers Using Hybrid DEA-OPA Model: A Sustainable Development Perspective . Group Decision and Negotiation . 31 . 2 . 335–362 . 10.1007/s10726-021-09770-x. 254498857 .
  46. Irvanizam . Irvanizam . Zulfan . Zulfan . Nasir . Puti F. . Marzuki . Marzuki . Rusdiana . Siti . Salwa . Nany . 2022 . An Extended MULTIMOORA Based on Trapezoidal Fuzzy Neutrosophic Sets and Objective Weighting Method in Group Decision-Making . live . IEEE Access . 10 . 47476–47498 . 10.1109/access.2022.3170565 . 2169-3536 . https://web.archive.org/web/20220923134405/https://ieeexplore.ieee.org/document/9763528/ . 2022-09-23 . 2022-09-19 . 248698791. free .
  47. Mahmoudi . Amin . Abbasi . Mehdi . Yuan . Jingfeng . Li . Lingzhi . 7 September 2022 . Large-scale group decision-making (LSGDM) for performance measurement of healthcare construction projects: Ordinal Priority Approach . Applied Intelligence . 52 . 12 . 13781–13802 . 10.1007/s10489-022-04094-y . 9449288 . 36091930.
  48. Web site: Web-based solver . 2022-10-15 . ordinalpriorityapproach.com.
  49. Web site: Matlab-based solver . 2022-10-15 . www.mathworks.com . en.