Finding the school and degree program that are right for you is one of the most important things you’ll ever do. It is a complex process that will affect the rest of your life.
To navigate this process successfully, you need useful and reliable information about the enormous variety of post-secondary schools and programs that exist in this country. That is why TheBestSchools.org provides a wide range of articles on many different aspects of higher education.
However, one of the things uppermost in your mind as you look over our site is bound to be the question of rankings. You want to know how some particular colleges or degree programs that interest you stack up against other similar schools and programs.
Therefore, we know that advice about rankings is one of the most important services we can provide to you. We also know that you have many choices among the numerous other ranking publications and websites that are out there.
So, why should you rely on our rankings, rather than someone else’s? And what is our ranking methodology, anyway?
To answer these important questions, let’s think for a moment about what a print publication or website really means when they say that a school or a degree program is among, say, the “top 10” in the country.
While actual rankings may differ considerably, a common method underlies most rankings, namely, “multicriteria optimization.” This method is practiced by the most famous ranking organizations out there, like U.S. News & World Report and The Princeton Review.
Multicriteria optimization works as follows. Having laid out the various criteria one regards as relevant to determining a school’s quality, one evaluates a college or degree program with respect to each of these criteria, and then pools all this information into a single measure, which then yields a ranking.
Because typically there are many such criteria, the information collected from these various criteria will by itself not be sufficient to form a ranking. That’s because different criteria may give conflicting answers. For instance, a school may rank highly for academics but miserably for quality of campus life. Which of these is more important and how should they be weighted in assigning a ranking?
What’s needed, then, once performance on all these criteria is evaluated, is some way of combining all these performance measures into a single number that best reflects how all the various criteria are to be taken into consideration at once. In this way one optimizes across the various criteria—hence “multicriteria optimization.”
What are some of these criteria? Academic excellence figures prominently among them. In this category, ranking organizations often solicit the opinions of a large number of trusted sources, like students, faculty, administration, alumni, and prospective employers. These opinions may be gathered by telephone interviews or by questionnaires. Less subjective ways of assessing academic excellence are also possible, such as scholarly output and impact of a school’s faculty as measured through publications.
What about criteria other than academic excellence? The best-known ranking organizations consider many other factors, such as tuition costs, availability of financial aid, locale, student demographics, student satisfaction, sports opportunities, natural beauty, partying potential, safety, and many others.
This does not mean that all the criteria are treated equally, however. Rather, each ranking organization will not only have its own methodology that lays out the primary criteria and ways of gathering the statistics for each of these, but it will also have its own private formula of how much “weight” to give to each of the criteria when the time comes to crunch the numbers and arrive at an overall estimation of a given college or degree program.
While the criteria themselves and the statistical methods underlying them may seem highly scientific, often these relative “weights” assigned to the various criteria can be quite subjective. Indeed, the method for assigning the weights may be anything but scientific. And yet this relative weighting of the criteria is the most important aspect of the overall calculation from a mathematical point of view.
Nevertheless, concerns about the objectivity of the weighting method aside, it may seem at first glance that the multicriteria approach is still the best way to assign an overall ranking to a set of colleges or degree programs. It is the only method that attempts to come up with a unique ranking based upon a broad spectrum of criteria considered all together at the same time.
But is that really the best way to go? We believe that, in most cases, it is not. The reason is that different students will have different criteria for what they regard as most important in their education. And this means that different students will differ in the rankings that are meaningful to them.
Some students may be mainly interested in academic excellence. Others may want to be assured of a good return on their very substantial financial investment. And still others may put a premium on congenial surroundings in which to live for the next four years.
The trouble with the multicriteria optimization approach is that all the variously weighted criteria dilute each other in such a way that the resulting ranking may not be optimal for anyone at all.
Now, one obvious way around this difficulty is to limit one’s ranking to a small number of criteria, or even a single criterion. This is what Forbes magazine and Payscale.com do, for example. They perform a straightforward calculation of “return on investment,” which can tell you how much extra salary, on average, you can expect to earn from a degree from a given institution, over and above what you could have expected to earn without it, as a function of the cost of the degree.
This type of ranking may be ideal for you—if, for example, return on investment is your overriding concern in choosing a college or degree program. However, if other factors are of equal or greater importance to you, then obviously the Forbes ranking will not answer your needs very well.
We believe the multicriteria approach to ranking practiced by U.S. News and others is too broad. And we also believe the unicriterion approach practiced by Forbes is too narrow, at least for most students. That is why we have chosen to follow a middle path between these two extremes.
Our ranking methodology is based on three broad master criteria, but the application of each of these criteria will vary from case to case, depending on the kind of college or degree program that is under discussion in a given article.
Here are our three master criteria:
- Academic excellence
- Return on investment
- Incidental benefit
What do these criteria mean? Let us briefly look at each one.
Academic excellence means, above all, the quality and productivity of the faculty. And how are the quality and productivity of the faculty measured? By intellectual reputation and influence within the wider academic community. But what are intellectual reputation and influence based on?
This is where things get a little bit complicated. To some extent, intellectual reputation and influence can be objectively measured, by such indicators as published books and papers, citations, prizes, patents, public visibility, and other similar factors.
Not entirely, however. Unfortunately, some academic disciplines today have become infected with intellectual superficiality and faddishness. Therefore, to arrive at a just evaluation of particular colleges and degree programs, purely quantitative measures of faculty productivity (number of publications, impact factors of journals, h-index and the like) may sometimes need to be adjusted in favor of work that is likely to be of deep and lasting value. We will make such adjustments wherever necessary.
The second master criterion is return on investment. By this, we mean the kinds of considerations that Forbes takes into account, which were already discussed above—namely, factors like employability and average starting salary that determine how quickly your investment in your college education will be amortized, and then how much better you may expect to do financially over the course of your life as a result of that investment than you could have expected to do without it.
The third master criterion is incidental benefit. By this we mean all those little, and not-so-little, things that have nothing directly to do with your education or future prospects, but may significantly impact your quality of life during your studies. This master criterion will include such factors as locale (urban vs. rural), student demographics (income, geographic, ethnic, etc.), student satisfaction (graduation rates), sports opportunities, natural beauty, campus safety, and many more.
Finally, how we apply the three master criteria will be determined on a case-by-case basis. For some types of ranking articles, academic excellence may predominate. For others, return on investment or incidental benefit may be more important to the type of ranking study we are conducting. In some cases, we may apply all three master criteria to a given ranking.
Crucial for any ranking methodology is transparency: transparency about the criteria being employed and transparency about how those criteria are weighted in computing the ranking. At TheBestSchools.org, we are committed to such transparency.
Primary and Secondary Education
When TheBestSchools.org began to publish school rankings, its focus was on higher education. Over time, however, we have expanded our focus also to include primary and secondary school education. Although many of the considerations guiding us in the ranking of colleges, universities, and their degree programs continue to apply at the primary and secondary level, not all do. What follows, then, is a brief clarification of the similaries and differences in our ranking methodology as it applies to primary and secondary education:
- Rankings at the primary and secondary level continue to employ a multicriteria optimization approach, which introduces unavoidable judgment calls regarding how the various criteria are weighted. Briefly put, no change here from what we described for higher education.
- Academic excellence continues to be our master criterion par excellence. TheBestSchools.org does not want to recommend a school lacking solid academics.
- What constitutes academic excellence at the primary and secondary school level, however, will differ a bit from academic excellence in higher education. Of course, there will be considerable overlap. Quality of faculty—their knowledge, ability to teach, and record of accomplishment—continue to be crucially important.
- Yet how faculty quality is gauged will differ between higher education on the one hand, and primary or secondary education on the other. We believe it helps when college and university faculty are actively engaged in research and thereby moving the frontiers of knowledge forward. In our view, it makes them (or should make them) more exciting teachers, letting students look over their shoulders in the process of discovery.
- But for primary and secondary education, teachers are not expected to make original contributions to a field of study. Rather, they will be teaching a substantial number of classes, and their job is to teach well without necessarily engaging in original research or writing. So, for primary and secondary education, this master criterion gets cashed out principally in terms of teachers being masters of their craft in the art of teaching.
- Although quality of teachers/faculty is an overriding concern in our ranking of higher education programs along the master criterion of academic excellence, academic excellence at the primary and secondary school level involves also a closer attention to the quality of students. Are the students learning well? Are they scoring well on tests? Is the life of the mind something the students at the school value? Are they challenging each other to learn? These considerations become more important in our view at the primary and secondary level than in higher education, where students typically are there at their own choice and also have gone through an admission process that is to varying degrees stringent.
- Our master criterion of return on investment also needs to be understood differently at the primary and secondary level. In higher education, return on investment is readily evaluated in terms of job prospects and salary. At the primary and secondary level, however, students are not looking to jump directly into a career (indeed, if they quit school at that level to pursue a “career,” it typically would be regarded as a failure of the school in not seeing them get their diploma and pursue further training either at college or in a trade).
- So, return on investment at the primary and secondary level is mainly a matter of preparing students to move on successfully to the next level in their education. Of course, for private primary and secondary education, affordability factors in here as well (a school that prepares students well for the next level of their education but is more affordable than another is, other things being equal, to be preferred).
- Our third master criterion of incidental benefit is, at the primary and secondary level, perhaps best reformulated in terms of learning enhancement. In other words, the question here becomes what advantages does the primary or secondary school provide to make the learning experience more effective and accessible. Smaller class sizes, healthy discipline, parental involvement, absence of bullying, safe buildings and grounds, etc. fall under this.
- Finally, in many of our rankings at the primary and secondary level, we employ diversity as a criterion (we use this criterion much less so for higher education because higher education, by its very nature, is typically able to draw from students across America and the world and thus tends to have a “built-in diversity actuator”).
- Often diversity at the primary and secondary level takes the form of geographical diversity—we prefer in our rankings not to have all the schools clustering in one or a few geographical areas. Diversity can also reflect on the ethnic diversity of a student body, or even its openness in bringing in foreign exchange students. It can involve exposing students to other cultures by taking trips abroad.
- In ranking primary and secondary schools, TheBestSchools.org attempts to underscore the shining lights in American education wherever they may be found. Diversity is thus, if you will, a fourth master criterion for us in the ranking of primary and secondary schools.