College selection: getting reliable information about colleges: Difference between revisions
No edit summary |
|||
| (8 intermediate revisions by one other user not shown) | |||
| Line 3: | Line 3: | ||
* Information about a college is often unreliable. | * Information about a college is often unreliable. | ||
* It is easier to get reliable information about hard-to-fudge metrics than about subjective experiences. | * It is easier to get reliable information about hard-to-fudge metrics than about subjective experiences. | ||
* The best source of reliable information about subjective experiences and for getting the inside scoop on easy-to-fudge metrics, as of now, appears to be [ | * The best source of reliable information about subjective experiences and for getting the inside scoop on easy-to-fudge metrics, as of now, appears to be [http://www.collegeconfidential.com/ College Confidential]. | ||
==Reasons why information about colleges is unreliable== | |||
* '''College incentives''': College admissions and outreach officers have incentives to portray themselves favorably to prospective students to maximize the number of student applications. | |||
* '''Lack of transparency makes it easy for colleges to fudge numbers''': Much of the data that colleges release cannot be independently verified by third parties, and colleges can therefore tweak the numbers in suitable ways. Outside agencies that rank and rate colleges often rely on this data and this corrupts their ratings. Some of these outside agencies rely on colleges to provide them information, and therefore need to maintain good relationships with the colleges. | |||
* '''Bias among other information sources''': Current and former students and others who give feedback about the college experience may be biased. Biases include selection bias (not everybody gives feedback, and those who do are atypical), endowment effect bias (current and former students are loyal to their alma maters because they want to feel they made the right decision), and sour grapes (people who were rejected from a college may want to believe the college was not good anyways). | |||
* '''Arbitrary weights''': Outside agencies that rank and rate colleges often use arbitrary "weights" when deciding how to combine different aspects of a college experience. Even if the individual measurements were correct, the weighted rating is not meaningful. | |||
{{quotation|'''KEEP IN MIND''': In general, information is usually not designed to be as accurate as possible. Think critically about both the '''incentives''' of people to be reliable and the '''flexibility''' they have in fudging or massaging the data.}} | |||
==Typical sources of information about colleges== | ==Typical sources of information about colleges== | ||
===College visits=== | ===College visits=== | ||
These include: | These include: | ||
| Line 21: | Line 28: | ||
===Guidebooks and their online equivalents=== | ===Guidebooks and their online equivalents=== | ||
Guidebooks are plagued by many of the unreliability problems highlighted above. | |||
The [http://www.amazon.com/The-Fiske-Guide-Colleges-2014/dp/1402260644 Fiske Guide to Colleges 2014] describes its methodology: | The [http://www.amazon.com/The-Fiske-Guide-Colleges-2014/dp/1402260644 Fiske Guide to Colleges 2014] describes its methodology: | ||
| Line 45: | Line 36: | ||
College administrators are motivated to portray their universities favorably, and so one would expect them to recruit students who are most likely to say good things about their universities. So one can't trust the students' responses to be representative of what it's like to be a student at the college. | College administrators are motivated to portray their universities favorably, and so one would expect them to recruit students who are most likely to say good things about their universities. So one can't trust the students' responses to be representative of what it's like to be a student at the college. | ||
=== | ===Independent forums for current and prospective students to interact with one another=== | ||
The best such source as of now appears to be [http://www.collegeconfidential.com College Confidential]. | |||
Given its growing role as a general repository of information in the question-and-answer format, [http://www.quora.com Quora] is a resource that might become increasingly valuable, although colleges ''per se'' are not Quora's focus at all. | |||
Latest revision as of 19:30, 24 February 2014
Key points
- Information about a college is often unreliable.
- It is easier to get reliable information about hard-to-fudge metrics than about subjective experiences.
- The best source of reliable information about subjective experiences and for getting the inside scoop on easy-to-fudge metrics, as of now, appears to be College Confidential.
Reasons why information about colleges is unreliable
- College incentives: College admissions and outreach officers have incentives to portray themselves favorably to prospective students to maximize the number of student applications.
- Lack of transparency makes it easy for colleges to fudge numbers: Much of the data that colleges release cannot be independently verified by third parties, and colleges can therefore tweak the numbers in suitable ways. Outside agencies that rank and rate colleges often rely on this data and this corrupts their ratings. Some of these outside agencies rely on colleges to provide them information, and therefore need to maintain good relationships with the colleges.
- Bias among other information sources: Current and former students and others who give feedback about the college experience may be biased. Biases include selection bias (not everybody gives feedback, and those who do are atypical), endowment effect bias (current and former students are loyal to their alma maters because they want to feel they made the right decision), and sour grapes (people who were rejected from a college may want to believe the college was not good anyways).
- Arbitrary weights: Outside agencies that rank and rate colleges often use arbitrary "weights" when deciding how to combine different aspects of a college experience. Even if the individual measurements were correct, the weighted rating is not meaningful.
KEEP IN MIND: In general, information is usually not designed to be as accurate as possible. Think critically about both the incentives of people to be reliable and the flexibility they have in fudging or massaging the data.
Typical sources of information about colleges
College visits
These include:
- Informal visit to a friend or sibling studying in the college where you might tag along with the friend to sit in classes and go to the other places the friend is going.
- Formal visits that colleges organize for admitted students, typically by having current undergraduates host them and by having them sit in classes based on a formal selection process.
- Brief visits where you are shown around the campus by an official tour guide, but don't sit in classes.
(1) is the most informative, (2) is less informative (because the college plays a role in scripting the experience more carefully in order to portray itself positively), and (3) provides very little information -- tour guides generally focus on describing the physical location and its historical significance rather than provide accurate information about the current student experience.
Guidebooks and their online equivalents
Guidebooks are plagued by many of the unreliability problems highlighted above.
The Fiske Guide to Colleges 2014 describes its methodology:
Administrators were ... asked to recruit a cross section of students to complete another electronic questionnaire with questions relating to what it is like to be a student at their particular college or university.
College administrators are motivated to portray their universities favorably, and so one would expect them to recruit students who are most likely to say good things about their universities. So one can't trust the students' responses to be representative of what it's like to be a student at the college.
Independent forums for current and prospective students to interact with one another
The best such source as of now appears to be College Confidential.
Given its growing role as a general repository of information in the question-and-answer format, Quora is a resource that might become increasingly valuable, although colleges per se are not Quora's focus at all.