The University Ranking by Academic Performance[1] (URAP) is a university ranking developed by the Informatics Institute[2] of Middle East Technical University. Since 2010, it has been publishing annual national[3] and global[4] college and university rankings for top 2000 institutions. The scientometrics measurement of URAP is based on data obtained from the Institute for Scientific Information via Web of Science and inCites. For global rankings, URAP employs indicators of research performance including the number of articles, citation, total documents, article impact total, citation impact total, and international collaboration. In addition to global rankings, URAP publishes regional rankings for universities in Turkey using additional indicators such as the number of students and faculty members obtained from Center of Measuring, Selection and Placement ÖSYM.
URAP gathers data from international bibliometric databases such as Web of Science and InCites provided by the Institute for Scientific Information. URAP uses data of 2,500 Higher Education Institutions (HEI) with highest number of articles published. The overall score of each HEI is based on its performance over several indicators. Of 2500 selected HEIs, the top 2000 are included in the rankings published by URAP. Field based rankings are performed on 23 fields based on Australia ERA.[5]
URAP uses 6 main indicator to measure the academic performance. These indicators are number of articles, citation, total documents, article impact total, citation impact total, and international collaboration. The raw bibliometric data underlying URAP's 6 main indicators have highly skewed distribution. To address this issue, the median of the indicators have been used. The Delphi system was conducted with a group of experts to assign weighting scores to the indicators. Total score of 600 is distributed to indicators. URAP uses additional indicators for ranking universities in Turkey including the number of students and faculty members.The following table shows the indicators used for global rankings in URAP as of 2014.
| Indicator | Objective | Weight (out of 600) | Source | |
|---|---|---|---|---|
| Number of Articles | Scientific Productivity | %21 | InCites | |
| Citation | Research Impact | %21 | InCites | |
| Total Documents | Scientific Productivity | %10 | InCites | |
| Article Impact Total | Research Quality | %18 | InCites | |
| Citation Impact Total | Research Quality | %15 | InCites | |
| International Collaboration | International Acceptance | %15 | InCites |
Number of articles is used as a measure of current scientific productivity which includes articles indexed by Web of Science. This indicator covers articles, reviews and notes. The weight of this indicator in the overall ranking is %21.
Citation, as an indicator in URAP ranking, is a measure of research impact. It is scored according to the total number of citations received. The weight of this indicator in the overall ranking is %21.
Total documents is the measure of sustainability and continuity of scientific productivity. The total document count covers all scholarly literature provided by the Web of Science database, including conference papers, reviews, letters, discussions, scripts, and journal articles. The weight of this indicator in the overall ranking is %10.
Article Impact Total (AIT) is a measure of scientific productivity adjusted by the ratio of institution's Citation Per Publication (CPP) to the world CPP in 23 subject areas. The ratio of the institution's CPP and the world CPP indicates whether the institution is performing above or below the world average in that field. This ratio is multiplied by the number of publications in that field and then summed across the 23 fields, as shown in the following formula:
AIT=
| 23 | |
| \sum | |
| i=1 |
[(
| CPPi | ||||
|
)*Articlesi]
The weight of this indicator in the overall ranking is %18.
Citation Impact Total (CIT) is a measure of research impact corrected by the institution's normalized CPP with respect to the world CPP in 23 subject areas. The ratio of the institution's CPP and the world CPP indicates whether the institution is performing above or below the world average in that field. This ratio is multiplied by the number of citations in that field and then summed across the 23 fields, as shown in the following formula:
CIT=
| 23 | |
| \sum | |
| i=1 |
[(
| CPPi | ||||
|
)*Citationsi]
The weight of this indicator in the overall ranking is %15.
International Collaboration is a measure of global acceptance of the institution. International collaboration data, which is based on the total number of published studies conducted in collaboration with foreign universities, is obtained from InCites. The weight of this indicator in the overall ranking is %15.
| Institution | 2021-22[6] | 2020-21[7] | 2019–20[8] | 2018–19[9] | 2017–18[10] | 2016–17[11] | 2015–16[12] | 2014–15[13] | |
|---|---|---|---|---|---|---|---|---|---|
 Harvard University | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | |
 University of Toronto | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | |
 University College London | 3 | 4 | 3 | 5 | 6 | 5 | 6 | 6 | |
| Stanford University | 4 | 3 | 4 | 4 | 5 | 4 | 8 | 7 | |
| University of Oxford | 5 | 5 | 5 | 3 | 3 | 3 | 3 | 3 | |
| Johns Hopkins University | 6 | 6 | 6 | 6 | 8 | 6 | 4 | 4 | |
| University of Cambridge | 7 | 7 | 7 | 7 | 9 | 8 | 5 | 5 | |
| University of Michigan | 8 | 8 | 8 | 9 | 11 | 10 | 10 | 10 | |
| University of Washington | 9 | 10 | 9 | 10 | 12 | 11 | 11 | 11 | |
 Tsinghua University | 10 | 12 | 12 | 18 | 25 | 38 | 48 | 58 | |
| Shanghai Jiao Tong University | 11 | 18 | 19 | 24 | 32 | 39 | 50 | 59 | |
| Imperial College London | 12 | 13 | 13 | 11 | 16 | 15 | 15 | 15 | |
 University of Paris-Saclay | 13 | 9 | 94 | 91 | 80 | 87 | 70 | 69 | |
| University of Pennsylvania | 14 | 15 | 14 | 15 | 14 | 13 | 13 | 13 | |
| Zhejiang University | 15 | 20 | 20 | 31 | 33 | 34 | 42 | 46 | |
| University of California, Los Angeles | 16 | 16 | 15 | 13 | 13 | 12 | 12 | 12 | |
| Massachusetts Institute of Technology | 17 | 11 | 11 | 8 | 7 | 7 | 7 | 9 | |
| Sorbonne University | 18 | 14 | 10 | 17 | 4 | 26 | 26 | 25 | |
| Columbia University | 19 | 17 | 16 | 14 | 15 | 14 | 14 | 14 | |
 University of Sydney | 20 | 24 | 24 | 23 | 26 | 27 | 29 | 30 | |
 University of Copenhagen | 21 | 19 | 18 | 16 | 17 | 16 | 16 | 22 | |
| Peking University | 22 | 21 | 21 | 22 | 29 | 33 | 44 | 48 | |
| University of Melbourne | 23 | 25 | 23 | 26 | 30 | 31 | 30 | 29 | |
| University of Paris | 24 | 23 | 97 | 95 | 87 | 85 | 78 | 76 | |
| University of California, San Diego | 25 | 26 | 22 | 19 | 18 | 17 | 17 | 16 | |
| University of California, San Francisco | 26 | 30 | 31 | 28 | 24 | 22 | 19 | 18 | |
 National University of Singapore | 27 | 32 | 28 | 30 | 27 | 29 | 32 | 34 | |
| Cornell University | 28 | 28 | 29 | 25 | 23 | 25 | 25 | 24 | |
| University of British Columbia | 29 | 29 | 27 | 27 | 21 | 21 | 22 | 20 | |
| Yale University | 30 | 27 | 26 | 21 | 20 | 19 | 20 | 21 | |
 University of São Paulo | 31 | 33 | 33 | 38 | 36 | 40 | 35 | 31 | |
| Monash University | 32 | 37 | 39 | 46 | 53 | 57 | 62 | 64 | |
| University of California, Berkeley | 33 | 22 | 17 | 12 | 10 | 9 | 9 | 8 | |
| University of Queensland | 34 | 35 | 35 | 39 | 40 | 41 | 43 | 51 | |
| Sun Yat-sen University | 35 | 48 | 60 | 83 | 93 | 99 | 113 | 116 | |
| Duke University | 36 | 34 | 34 | 33 | 28 | 24 | 24 | 23 | |
 University of Tokyo | 37 | 31 | 25 | 20 | 19 | 18 | 18 | 17 | |
 KU Leuven | 38 | 38 | 37 | 42 | 41 | 23 | 23 | 38 | |
| University of New South Wales | 39 | 42 | 44 | 52 | 60 | 71 | 74 | 78 | |
 University of Amsterdam | 40 | 40 | 48 | 51 | 63 | 61 | 61 | 61 | |
| Ohio State University | 41 | 39 | 38 | 36 | 38 | 37 | 33 | 32 | |
| Huazhong University of Science and Technology | 42 | 49 | 56 | 81 | 94 | 111 | 135 | 155 | |
| University of Pittsburgh | 43 | 43 | 40 | 35 | 35 | 32 | 31 | 28 | |
 Karolinska Institute | 44 | 50 | 51 | 48 | 51 | 53 | 60 | 57 | |
| Fudan University | 45 | 54 | 58 | 62 | 68 | 74 | 90 | 94 | |
| University of Chicago | 46 | 36 | 30 | 29 | 22 | 20 | 21 | 19 | |
| Northwestern University | 47 | 46 | 45 | 43 | 46 | 38 | 37 | 37 | |
| Utrecht University | 48 | 45 | 42 | 40 | 45 | 44 | 37 | 35 | |
 Seoul National University | 49 | 51 | 47 | 49 | 44 | 50 | 36 | 42 | |
| University of Minnesota | 50 | 44 | 32 | 32 | 31 | 28 | 27 | 27 |
URAP covers considerably more institutions than other major ranking systems. In a section about URAP in “Where Are the Global Rankings Leading Us? An Analysis of Recent Methodological Changes and New Developments” published in the European Journal of Education it is mentioned that ”While it is less well-known than SRG, ARWU, THE, and QS, it is interesting because it published a list of 2000 universities, while the above rankings cover a maximum of 700 universities.”[14] This is also mentioned in the “EUA report on Ranking for 2013 “ published by the European University Association. It indicates that URAP, along with SCImago ranking system, “fill an important gap in the rankings market in that their indicators measure the performance of substantially more universities, up to 2000 in the case of URAP and over 3000 in SCImago, compared to only 400 in THE, 500 in SRC ARWU, NTU ranking and CWTS Leiden, and around 700 in QS.”[15]
URAP is mentioned as one of the four ranking systems that solely measure the academic performance. The other three are Performance Ranking of Scientific Papers for World Universities, CWTS Leiden Ranking, and SCImago Institutions Rankings.[16] URAP excludes teaching indicators, such as student quality and teaching performance, from global rankings and only covers research-oriented indicators.[17] In the “International Benchmarking in UK higher Education”[18] report of the Higher Education Statistics Agency, URAP is listed among the benchmarking resources for measuring academic. In the same report, URAP is categorized in the “whole university rankings” along with Times Higher Education World University Rankings (THE), QS World University Rankings, Academic Ranking of World Universities (ARWU), CHE Excellence Rankings, RatER Global University Ranking of World Universities, Webometrics Ranking of World Universities, 2010 World University Ranking, SIR World Report, CWTS Leiden Ranking, U-Multirank, European Research Ranking, Performance Ranking of Scientific Papers for World Universities, Human Resources & Labor Review (HRLR), and Professional Classification of Higher Education Institutions.
URAP is mentioned and used in several studies based on, or referring to, global rankings. In the “World University Ranking Systems: An Alternative Approach Using Partial Least Squares Path Modeling” article, published in the Journal of Higher Education Policy and Management, Urap is incorporated in the suggested model as one of the nine major worldwide university ranking systems along with ARWU, QS, Times, Webometrics, Taiwan. Leiden, SIR, and CWUR. In the same article, URAP is categorized among the ranking systems that are based solely on publication performance. The other ranking systems in the same category are Performance Ranking of Scientific Papers for World Universities, CWTS Leiden Ranking, and SCImago Institutions Rankings.
The following is a list of some of the books, peer reviewed articles, and conference proceedings that have covered URAP or have incorporated it in their models or comparisons.
Annual URAP ranking results are used by a number of listed universities to indicate their academic performance. The following is a short list of links to university pages that has mentioned URAP results either independently or in conjunction with other ranking results.
The indicators used in URAP are absolute values and size-dependent making it biased towards larger institutions. According to the “EUA report on Ranking for 2013“ published by the European University Association, URAP disregards books, excludes studies in arts and humanities areas, and under-represents social sciences. Furthermore, URAP does not employ any compensation for different publication cultures due to the lack of field-normalization of the results of bibliometric indicators. The report further states that “The results of the indicator on citation numbers in particular, as well as those on publication counts, are thus skewed towards the natural sciences and especially medicine.” It also states that excluding teaching indicators by URAP makes its focus solely on research-oriented institutions.
The “University Ranking Lists: A directory” report published by the Division for Analysis and Evaluation of the University of Gothenburg points out a problem that might arise from including more than 500 institutions in the ranking system. It states that “It [URAP] lists 2000 universities, and the purpose is to provide a ranking that covers not only institutions in the Western elite group. This purpose contrasts starkly with other ranking producers’ decisions not to publish more than the 400-500 top positions of their lists, since they do not consider their methods reliable below that level. [URAP] do not comment this problem.”