<p>" I am at least somewhat curious. Why the drop at some schools?"</p>
<p>They've spent four years drinking themselves silly. That's my theory and I'm sticking to it.</p>
<p>LOL. That's a good theory. Unfortunately mathmom, I don't think my D's school is far behind any of them in that particular category.</p>
<p>Could another possibility be that some students at some schools are just 4 more years away from US and World History, and because of AP have zero intervening coursework in the area? </p>
<p>I know my D will have zero courses in US History or Politics in college if she follows through with her current plans. I assume that is not that unusual and if the test tested rote (memorized actor, treaty, date , place ) knowledge , I can see where she might perform lower as a senior. </p>
<p>Is that all this study is saying? Have more required civics type history and poli sci courses? (Like Grove City?) Who financed this thing? Seems there may be an agenda. (Like anyone doubted that after hearing about the "Crisis". ;))</p>
<p>I give the authors of this report an F in statistics and survey methodology, because they omitted some important caveats.</p>
<p>To measure "value-added" accurately requires a longitudinal study, where you sample the freshman class that entered in a given year, then resample the same class four years later.</p>
<p>They didn't do that. They sampled the freshman and senior class at the same instant in time. The study is a bit unclear about exactly when the survey took place, but let's say fall 2004. So they tested samples of the class of 2005 (seniors) and the class of 2008 (freshmen).</p>
<p>That is problematic because admission became increasingly selective during this period, so the freshmen who entered in 2004 might well have been better prepared than the seniors-in-2004 who had entered college in 2001. </p>
<p>Thus, the difference in freshman vs. senior scores might not measure value-added correctly because, e.g., perhaps an increasing proportion of freshmen in 2004 had taken AP US history and AP government compared to the seniors who had entered college three years earlier.</p>
<p>Another statistical issue: what statisticians call "survivorship bias." If 30% of entering students flunk out or drop out (which could well be the case at some schools on the list, e.g., Colorado State University, which has a 6-year graduation rate around 60%), then you would expect that the seniors in your sample (who are "survivors" of their entering cohort) will be stronger academically than a random sample of their entering class would have been four years earlier.</p>
<p>Thus, for both the above reasons, presenting statistics on simultaneous samples of the freshman and senior class from a single moment in time is a very poor measure of value-added.</p>
<p>EDIT: All statistical studies have flaws and it's never possible to get flawless data, but responsible statisticians point out the flaws in their approach and attempt to analyze how much error they might contribute. The authors of this study did not do that, as far as I can see. Thus, my grade of F for statistics.</p>
<p>Education Secretary Margaret Snellings is calling for, among other things, quantifiable data regarding the effectiveness of the education at various colleges that receive federal funds. </p>
<p>I shudder to think that this (no college student left behind?) would result in wholesale testing of the type of this civics test.</p>
<p>Does it stand to reason that schools that have a preponderance of math, science, and technical programs would fare worse than those schools that have more students in poli-sci, international studies, history, etc? My son is on a science/premed track and most of his classes are science and math, with humanities and social science courses on the side. If he were majoring in something in the social studies field, he would probably end up a lot more literate in civics by his senior year but thats not the focus of his major.</p>
<p>Civics is one facet of education. If they were to repeat the survey with a test on hard sciences or math, theyd probably end up with totally different results.</p>
<p>Im confused as well. What exactly is this supposed to prove?</p>
<p>I find it interesting that some of the best colleges enroll freshman students who graduate knowing about the same amount of basic history (or less) than when they enrolled. Maybe this is related to the fact that few of the top colleges require that students take US history.</p>
<p>
[quote]
Maybe this is related to the fact that few of the top colleges require that students take US history.
[/quote]
</p>
<p>And should they?</p>
<p>Larry Summers was upset that American students did not know enough science; others are upset that students don't know enough statistics. Still others are upset that college students cannot write; still others clamor for classes on ethics... </p>
<p>Most college students have taken US history classes in high school. But they have not taken classes in the history of the Middle East (and let's face it, in 2001, were most Americans concerned about the Ba'athist Party?) or of other parts of the world. So while many students will take a history class in order to fulfill their gen ed requirements, unless they want to have an easy time going over the history of the US once again, they will seek out courses in the history of different countries and times. As they should.</p>
<p>EDIT: Having said this, I have to disagree with the implications in DRJ's post. The students at the top colleges performed better on the test than students at lower ranked schools. By the terms of this flawed test, they have a better grasp of American history than others.</p>
<p>
[quote]
Maybe this is related to the fact that few of the top colleges require that students take US history.
[/quote]
</p>
<p>Few colleges of any sort require that students take US history. If colleges have a liberal arts distribution requirement for students to take history at all, it usually gives quite a bit of lattitude for choice--there is an option to satisfy the history requirement by taking a history course in a variety of different geographic settings and/or time periods. </p>
<p>Even Rhodes College doesn't seem to have an American history requirement as far as I can see. There is a history requirement but it appears there is flexibility in how to meet it, no requirement to take American history specifically. (Of course, it could turn out that the most popular, charismatic, and talented history professors at Rhodes teach American history, but Rhodes' success in this dubious measure of "value-added" doesn't seem to come from an American history requirement.)</p>
<p>I doubt that many of the other colleges on the top of the ISI survey list require American history either. I suspect their strong "value-added" scores are artifacts of survivorship bias, since most have poor retention ratios.</p>
<p>Rhodes does not fit that pattern, however. They have a very strong retention rate, so survivorship bias does not account for their strong showing.</p>
<p>For those interested in the kinds of questions typically asked by such surveys, here is an older but similar survey published a few years ago. (This survey only looked at college seniors at top-ranked schools, top 25 universities and top 25 LACs according to USN&WR. They performed even more abysmally on this test than the 2004 seniors did on the ISI test.) The appendix also gives details of which top-ranked colleges require history at all, let alone American history. According to their analysis, 22% of those top colleges required history, but none required American history.</p>
<p>
marite, as the "data" (such as it is ) shows that's generally true if you don't consider Michigan, Texas, Cal, Duke, UNC, Cornell, Georgetown and Brown to be in your category of top colleges. (They make mine.) Their seniors averaged less than 60%. What you say is true of HYPSM but that's about as far as it goes among tested schools.</p>
<p>Cur:</p>
<p>here's the list of schools whose freshman class scored 60 or above. Cornell is out by a hair (59.4):</p>
<p>18 Princeton University 66.0 68.8 2.8
25 Harvard University 67.8 69.7 1.9
30 American University 63.4 64.4 1.0
31 Stanford University 62.2 63.1 0.9
33 Washington & Lee University 63.6 63.8 0.2
34 Dartmouth College 67.9 68.0 0.1
37 University of Chicago 64.5 64.2 -0.3
38 Massachusetts Inst. of Technology 63.9 63.5 -0.4
39 Williams College 69.1 68.4 -0.7
42 University of Virginia 63.7 62.6 -1.1
43 Georgetown University 69.1 67.9 -1.2
44 Yale University 69.8 68.3 -1.5
46 Duke University 60.6 58.3 -2.3
47 Brown University 62.5 59.8 -2.7
49 University of California, Berkeley 60.4 54.8 -5.6
50 Johns Hopkins University 61.7 54.4 -7.3</p>
<p>My remarks had to do with admissions rather than the supposed value added of 4 years of college. For the reasons adumbrated by wisteria, I discount the statistics on seniors.</p>
<p><adumbrated>
I learned a new vocab word today! :):)</adumbrated></p>
<p>Thank you wisteria! I think you pretty much covered my points, but I will make them anyway!! :)</p>
<p>I have seen so many flawed "studies" announced with press releases that I am no longer shocked when I see them, even when they are produced by university social science/research departments. This study surveyed freshmen and seniors, and then attributed to the difference to the college itself. How silly! Maybe the current freshman class is smarter than the current senior class was when they were admitted. The survey designers could have at the very least controlled for average SAT scores. Maybe college kids are not exposed to network news the way HS kids are who live at home. So a senior might know less about current affairs after 4 years in the ivory tower than a fresh off the boat freshman. There are just so many alternate explanations when you have a sloppy survey.</p>
<p>BTW, the top schools for senior scores are in order:
1 Harvard
Princeton
Grove City
Yale
5 Dartmouth
Georgetown
American
Chicago
W+L
10 MIT
Stanford
UVA
Rhodes
Wheaton
15 Calvin
Brown
Duke
UNC
Cornell
20 GMU</p>
<p>I had forgotten Williams in the top 5. </p>
<p>marite, I know why someone might want the senior numbers to be disregarded (doesn't look good for some of the elites below HYPSM. Failing scores for 2 Ivies :eek:) but they are at least as valid as the numbers for the freshman class. I believe Wisteria was talking about the methodology comparing the two classes being faulty for the reasons she listed. ;)</p>
<p>That being said I have looked down the trail and it is: </p>
<p>a bunch of right-wing wackos</p>
<p>or</p>
<p>citizens concerned with the demise of "American Values" (whatever that is). </p>
<p>Take your partisan pick.</p>
<p>Choosing the Right College (edit: an ISI publication) is aimed…at exposing the political biases of academe, the prevalence of permissive sex and the lack of core curriculums to prospective students and their parents."
— New York Times </p>
<p>Quoted on the ISI website.</p>
<p>QUESTION: Is there any mandatory student orientation that exposes students to sexually explicit material or graphic explanations of sexual practices? </p>
<p>EXPLANATION: Films that most parents would consider pornographic are often shown during orientation. Practices that violate family morals may be presented in positive terms or even advocated.</p>
<p>Gee, I forgot to ask that one. </p>
<p>My Question: What are these people smoking?</p>
<p>the other factor that potentially looms large is AP credits, particularly at the publics. Most kids going to Cal have taken APUSH and AP Govt. A 3 passes one out of those classes and fulfills the distribution requirements at the big Uni. Thus, a student could easily graduate w/o taking a history class on campus. Ditto Brown, which, as we know, is develop your own major requirements. In both cases, Frosh will remember more of their HS history offerings, and may have forgotten it by senior year. </p>
<p>Whether college kids should be forced to take history is another issue, altogether.</p>
<p>Cur:</p>
<p>senior scores may be valid but not as an indication of value added by the colleges since colleges do not require US history as a graduation requirement. Currently, Harvard requires TWO "historical studies" classes for graduation (a requirement likely to be modified by the curricular review). At many other colleges, the graduation requirement calls for one class in history.
My S is contemplating two courses, neither of which has to do with US history. He did take APUSH, as, I imagine, did the majority of applicants to Harvard and other elite schools. Had he been tested as a freshman, he would probably have done very well. As a senior? I'm not sure. But I do not fault Harvard for his not remembering historical trivia from four years earlier, nor do I fault other colleges for their seniors not knowing much US history.<br>
All college students should know some science and some math, they should know how to write English properly, they should be able to think historically and critically. And that is different from being able to play Trivial Pursuit.</p>
<p><what are="" these="" people="" smoking?="">
I don't know, but I know I don't smoke anything and I was a bit taken aback by the orientation at my son's school (as was he) because they had to include safe sex, as well as sexual harrassment, acceptance of sexual diversity, blah, blah, blah. I have no clue why they think it's any of their business. Personally I think it's an invasion of kids' privacy. I would have found all those sex talks embarrassing and bizarre when I was that age. If a school has a huge interest in "orientating" the kids' attitudes toward sex, I would like to be aware of it.</what></p>
<p>Yesterday, I said I thought the methodology was OK. (I appreciated wisteria's point, but I don't think it's fair to ask researchers to take on a project that would require six years to complete, so I thought testing different seniors and freshmen would be acceptable. The bias wisteria identifies would tend to overstate improvement, which is a little scary given how little improvement there was.) But I thought more about it last night, and there is a significant sample-size problem.</p>
<p>They tested 1,400 students, which seems pretty good until you realize that there are 50 schools and two classes per school in the study. That means each class included only 14 kids. While I'm sure testing 14 randomly selected kids can tell you something -- after all, the order of the actual test results for freshmen looks awfully like the world as we know it -- the margin of error has to be fairly high. I suspect it's higher than the difference between freshmen and senior results at all but four or five of the schools, which pretty much vitiates any value of the study, even if you accept its goofy premise, unless you think it's valuable to confirm that none of the schools tested makes it a priority to educate all students about the subjects tested.</p>
<p>That still doesn't explain why seniors at Cal and Hopkins seem somewhat dumber than the freshmen. I lean towards the years-since-APUSH theory on that, but it may just be a random result.</p>
<p>By the way, if you look at the subjects tested, it was about 60% US history (split between events and theoretical background), 20% modern foreign policy issues (including historical subjects like Vietnam and the Cold War, and the famous question about Saddam Hussein's political base), and 20% basic economics). The subjects don't look crazy. I am not happy that kids at top schools only got about two thirds of the answers right on average (although apparently 115 kids had perfect scores).</p>
<p>
[quote]
The bias wisteria identifies would tend to overstate improvement, which is a little scary given how little improvement there was.
[/quote]
</p>
<p>Actually, I identified two sources of bias. One tends to understate improvement. The other tends to overstate it.</p>
<p>Understatement bias comes from the fact that the 2004 admissions was more selective at many schools than admissions a few years earlier, due to demographic changes.</p>
<p>Overstatement bias comes from the "survivorship" effect, i.e., the fact that at a school with low retention rates, a random sample of seniors will not be statistically representative of the overall cross-section of freshmen who entered the school.</p>
<p>So the two biases cut in different directions. At any particular school, one bias may be stronger than the other. </p>
<p>
[quote]
(I appreciated wisteria's point, but I don't think it's fair to ask researchers to take on a project that would require six years to complete, so I thought testing different seniors and freshmen would be acceptable.
[/quote]
</p>
<p>I didn't ask the researchers to take on a six-year project, I simply stated that responsible statisticians would take the intiative to point out the potential flaws in their study and attempt some error analysis to adjust for those flaws. (For example, they could have appended a brief questionnaire asking students if they had taken AP US history and AP government, etc.)</p>
<p>In addition to concerns about sample size raised by JHS, a much bigger statistical concern is whether the samples were truly random and representative. They did not describe the sampling methodology. </p>
<p>As the famous case of the Literary Digest poll revealed in the 1936 presidential elections, even a sample size of 10 million will not be statistically reliable unless it is representative of the overall population. Ever since then, responsible statisticians describe their sampling method in detail to show that they have taken steps to overcome this potential source of bias.</p>
<p>JHS:</p>
<p>The two history courses my S plans on taking have to do with Renaissance history, including one about the Scientific Revolution. He did not take AP-Euro in high school. He sees no reason to re-take American history since he took APUSH. He is not a history buff. So unless he were able as a senior to recall what he studied as a 10th grader, he would likely fail at 80% of the test.
A fair test of what is learned in college? You be the judge.</p>