College selection: getting reliable information about colleges

Key points

• Information about a college is often unreliable.
• It is easier to get reliable information about hard-to-fudge metrics than about subjective experiences.
• The best source of reliable information about subjective experiences and for getting the inside scoop on easy-to-fudge metrics, as of now, appears to be College Confidential.

Reasons why information about colleges is unreliable

• College admissions and outreach officers have incentives to portray themselves favorably to prospective students to maximize the number of student applications.
• Much of the data that colleges release cannot be independently verified by third parties, and colleges can therefore tweak the numbers in suitable ways. Outside agencies that rank and rate colleges often rely on this data and this corrupts their ratings. Some of these outside agencies rely on colleges to provide them information, and therefore need to maintain good relationships with the colleges.
• Current and former students and others who give feedback about the college experience may be biased. Biases include selection bias (not everybody gives feedback, and those who do are atypical), endowment effect bias (current and former students are loyal to their alma maters because they want to feel they made the right decision), and sour grapes (people who were rejected from a college may want to believe the college was not good anyways).
• Outside agencies that rank and rate colleges often use arbitrary "weights" when deciding how to combine different aspects of a college experience. Even if the individual measurements were correct, the weighted rating is not meaningful.

KEEP IN MIND: In general, information is usually not designed to be as accurate as possible. Think critically about the reliability of a given source.

Some reasons why information about what a given college is like is often misleading are:

Distortionary marketing

A college's promotional materials and admissions officers are generally distortionary: they exaggerate the college's positive qualities while omitting discussion of its negative qualities.

Unreliable college guidebooks

The Fiske Guide to Colleges 2014 describes its methodology:

Administrators were ... asked to recruit a cross section of students to complete another electronic questionnaire with questions relating to what it is like to be a student at their particular college or university.

College administrators are motivated to portray their universities favorably, and so one would expect them to recruit students who are most likely to say good things about their universities. So one can't trust the students' responses to be representative of what it's like to be a student at the college.

Current students and alumni being biased in favor of their schools

After somebody makes a choice, he or she generally wants to believe that he or she made the right choice. So people who have chosen to attend a college will often exaggerate its virtues and minimize its negative qualities.

Sour grapes

People who are rejected from a given college, or who don't apply because they know they can't get in, will sometimes have a "sour grapes" reaction, and be motivated to believe that the college isn't so good.

Typical sources of information about colleges

College visits

Further information: College visit

These include:

1. Informal visit to a friend or sibling studying in the college where you might tag along with the friend to sit in classes and go to the other places the friend is going.
2. Formal visits that colleges organize for admitted students, typically by having current undergraduates host them and by having them sit in classes based on a formal selection process.
3. Brief visits where you are shown around the campus by an official tour guide, but don't sit in classes.

(1) is the most informative, (2) is less informative (because the college plays a role in scripting the experience more carefully in order to portray itself positively), and (3) provides very little information -- tour guides generally focus on describing the physical location and its historical significance rather than provide accurate information about the current student experience.

Independent forums for current and prospective students to interact with one another

The best such source as of now appears to be College Confidential.