Comments on: Global university rankings - the best of all possible worlds? http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/ Tue, 13 May 2008 08:19:31 +0000 http://wordpress.org/?v=2.5.1 By: Simon Marginson http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-69 Simon Marginson Sat, 15 Mar 2008 03:34:13 +0000 http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-69 Kenny is right to spot the key problem with university rankings metrics (that they eliminate contextual data that enable us to interpret the numbers) but, I think, less convincing in advocating reputational surveys as the preferred alternative. All the evidence we have on reputational surveys suggest that most people who fill them in do not know the field of institutions well enough to make deep judgements. They only know institutions where they have worked or studied and this small group (possibily as small as one institution) is not enough. So if consumers are to be guided by rankings as Vicky suggests, it would be a case of 'the blind leading the blind'. I would be much more confident if rankings judgements were based on a study by an expert group which studied the 'outputs' of insttiutions in context, conducted case visits and drew on all available evidence. It would be an expensive process, perhaps too expensive to conduct annually, but it would produce universally valuable data and would develop a much stronger culture of scrutiny and product improvement. This in turn would lead to better judgements and decisions by consumers in the marketplace and that too would 'lift the bar'. Kenny is right to spot the key problem with university rankings metrics (that they eliminate contextual data that enable us to interpret the numbers) but, I think, less convincing in advocating reputational surveys as the preferred alternative. All the evidence we have on reputational surveys suggest that most people who fill them in do not know the field of institutions well enough to make deep judgements. They only know institutions where they have worked or studied and this small group (possibily as small as one institution) is not enough. So if consumers are to be guided by rankings as Vicky suggests, it would be a case of ‘the blind leading the blind’. I would be much more confident if rankings judgements were based on a study by an expert group which studied the ‘outputs’ of insttiutions in context, conducted case visits and drew on all available evidence. It would be an expensive process, perhaps too expensive to conduct annually, but it would produce universally valuable data and would develop a much stronger culture of scrutiny and product improvement. This in turn would lead to better judgements and decisions by consumers in the marketplace and that too would ‘lift the bar’.

]]> By: Kenny Easwaran http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-57 Kenny Easwaran Tue, 11 Mar 2008 22:21:30 +0000 http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-57 You bring up some important criticisms of reputational rankings (that they can be confounded by "brand" recognition, that they often have very low response rates, that they are self-reinforcing, and that they have no necessary connection to "fundamental capacity or performance") but it seems that in some cases, they are still better than the "quantitative" rankings that are often preferred. For instance, looking at citation numbers privileges achievement in some disciplines more than others. Graduation rates don't indicate whether students are very talented or whether the program is very easy to complete. And so on. In many ways, reputational rankings, when done very carefully (first of all, don't ask students to rank law schools!) may be much more relevant for many purposes. The people who are solicited for reputational rankings are hopefully the ones who know the most about different systems and can understand what features of the difference are relevant or not, while explicit quantitative rankings often fail to recognize these differences. Of course, the points about making specialist rankings, having many different ranking organizations, and making these organizations as disinterested as possible are very important points. You bring up some important criticisms of reputational rankings (that they can be confounded by “brand” recognition, that they often have very low response rates, that they are self-reinforcing, and that they have no necessary connection to “fundamental capacity or performance”) but it seems that in some cases, they are still better than the “quantitative” rankings that are often preferred. For instance, looking at citation numbers privileges achievement in some disciplines more than others. Graduation rates don’t indicate whether students are very talented or whether the program is very easy to complete. And so on. In many ways, reputational rankings, when done very carefully (first of all, don’t ask students to rank law schools!) may be much more relevant for many purposes. The people who are solicited for reputational rankings are hopefully the ones who know the most about different systems and can understand what features of the difference are relevant or not, while explicit quantitative rankings often fail to recognize these differences.

Of course, the points about making specialist rankings, having many different ranking organizations, and making these organizations as disinterested as possible are very important points.

]]> By: Vicky Phillips http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-26 Vicky Phillips Wed, 27 Feb 2008 17:51:27 +0000 http://www.ssrc.org/blogs/knowledgerules/2008/02/25/global-university-rankings-the-best-of-all-possible-worlds/#comment-26 The concept of "college rankings" looks very different depending on one's primary viewpoint. You've written an excellent piece when the issue is looked at from the perspective of an "inside academic." But what if this issue is viewed from the perspective of the student: especially a student in the USA who faces the daunting task of trying to locate that relatively new creature: the online degree. Potential learners, unlike academicians, love rankings, ratings and lists. This is because in the USA the choice of university porgrams, even online, is staggering: the cost likewise. Our firm, GetEducated.com, just completed our national (USA) rankings of online MBAs which are AACSB accredited. There exist 168 accredited MBA options in the USA: 62 of these AACSB accredited (which many consider the gold standard). Of these, the very same degree, the MBA, can cost from $8,000 to well over $100,000. Shocking, isn't it? What is the average potential learner to think of this wide a spread in cost? Is a $100,000 degree more credible than a $10,000 one? We all know there is no strong and fast relationship between cost and quality in higher ed (or a very shaky one at best in the USA!). Consumers deserve more transparency and explanation of why higher education presents at such wide price and prestige points. They turn to various ratings and ranking systems to determine what they might be buying (other than an early shot at bankruptcy) for such wide price points. There should be *more* ratings and rankings and such systems should use all types of reference viewpoints, especially those that matter to the learners themselves. Vicky Phillips ~ Chief Education Analyst - GetEducated.com The concept of “college rankings” looks very different depending on one’s primary viewpoint. You’ve written an excellent piece when the issue is looked at from the perspective of an “inside academic.”

But what if this issue is viewed from the perspective of the student: especially a student in the USA who faces the daunting task of trying to locate that relatively new creature: the online degree.

Potential learners, unlike academicians, love rankings, ratings and lists. This is because in the USA the choice of university porgrams, even online, is staggering: the cost likewise. Our firm, GetEducated.com, just completed our national (USA) rankings of online MBAs which are AACSB accredited. There exist 168 accredited MBA options in the USA: 62 of these AACSB accredited (which many consider the gold standard). Of these, the very same degree, the MBA, can cost from $8,000 to well over $100,000.

Shocking, isn’t it?

What is the average potential learner to think of this wide a spread in cost? Is a $100,000 degree more credible than a $10,000 one? We all know there is no strong and fast relationship between cost and quality in higher ed (or a very shaky one at best in the USA!).

Consumers deserve more transparency and explanation of why higher education presents at such wide price and prestige points. They turn to various ratings and ranking systems to determine what they might be buying (other than an early shot at bankruptcy) for such wide price points.

There should be *more* ratings and rankings and such systems should use all types of reference viewpoints, especially those that matter to the learners themselves.

Vicky Phillips ~ Chief Education Analyst - GetEducated.com

]]>