<p>It’s obvious how the anti-intellectual, faux-populist, chest-thumping American disregard for facts and figures is also deluding many into thinking that their problems could be simply wished away by character assassination of tables and statistics.</p>
<p>This thread was going along just fine until cellardweller ruined it with his “facts”. Don’t you know we all have our own unique creative viewpoints which are all equally legitimate, even if we didn’t look at the data?</p>
<p>^lol
</p>
<p>Please don’t drink the Koolaid. Education, eventually, DETERMINES the free market ;)</p>
<p>Screwitlah–apparently you are not reading my posts. I am in no way making excuses for the US. In fact, I am indicting it with a flawed social approach that pervasively undercuts the ability of part of our population to thrive in our society.</p>
<p>I may have come across as making excuses for US schools (I am not) or attacking statistics (I am not). I am questioning item response theory as a valid scoring technique. Please take a look at the assumptions on which it is based. I strongly prefer the more direct computation of scaled scores, as done on the PSAT and SAT, or simply measures of per cents correct, incorrect, and omitted.</p>
<p>The document referenced below gives an analysis of the old GRE Analytical section. This is the one that has questions such as: Tom is wearing a black down coat. Bill did not bicycle to class today. Jim and Dan have baseball caps on backwards. Jason lives in a suburb. The student who did bicycle to class ate at McDonald’s yesterday. Dan doesn’t take geology. The student who is wearing a windbreaker lives downtown in a loft . . .
Who took a limousine to class?</p>
<p>The GRE didn’t have an analytical section when I took it, and it has already been eliminated in favor of the analytical writing section. So I have no personal stake in this analysis. </p>
<p>However, take a look at the following document, and particularly the discussion of Table 4 on page 5. (Table 4 itself is at the end of the document, but it’s discussed on page 5.)</p>
<p><a href=“Psychology | College of Liberal Arts”>www.psych.umn.edu/psylabs/catcentral/pdf%20files/br02-01.pdf</a></p>
<p>This covers a case where two students have very similar performance going into item 28 of a 35 item test, with projected scores at that point of 670 and 640. Both answer the next 7 questions incorrectly (due to time pressure), and answer the final question correctly. One winds up with a 640 and the other with a 330.</p>
<p>I realize that the dramatic effect in this case is substantially due to the adaptive testing method on the GRE.</p>
<p>However, there is an important comment that applies to all item response theory scoring, with fixed questions as well as adaptive questions: namely, that IRT seems to have a difficult time scoring a student who gets difficult questions right and misses some easy questions. That is not supposed to happen. </p>
<p>There is no question that many US public schools, especially in areas of high poverty, do not actually deliver a competitive education.</p>
<p>So, let me consider a hypothetical suburban Massachusetts school, where the students might be expected to perform at least as well as in Shanghai. What might stop them? It seems to me that there are two main issues: </p>
<p>First, the students in Massachusetts might be as capable as the students in Shanghai if the exam did not have a time limit. Perhaps the Massachusetts students are slightly less quick to respond, and run into time pressure toward the end of the exam. This would result in leaving items blank or guessing randomly, both of which depress the scores. If one had simple % correct scoring, and particularly if one could look at the pattern at the beginning, middle, and end of the test, one could check easily for this possibility. It is obscured by the use of IRT.</p>
<p>Second, students with ADHD and other students who may make dumb mistakes (count me in the latter group, when I was a student) can also run afoul of the IRT presumption that students will get easy questions right. I’d like to know the impact of this on the scoring, before I placed too much credence in the outcome.</p>
<p>In my teaching, I have encountered a few students who are the IRT dream student: their answers are essentially 100% correct up to a certain difficulty level, and then they have nothing beyond that. This is on exams that have long problems to solve. Other students are all over the map, though. In fact, I would imagine that every individual has a personal profile of % correct vs. difficulty of question. The very bright ADHD crowd is likely to have about 85% correct on easier to moderate questions, but then to have a very long tail extending out to give correct answers to many very difficult questions. (If someone on this thread knows that IRT can handle scoring this group correctly, I’d appreciate a link to an article on the issue.) These students are the “deep, but quirky” thinkers that I had in mind before–my earlier phrasing may have suggested something more than that (e.g., mini-Einsteins), which I didn’t really intend.</p>
<p>"BBD</p>
<p>Here is the technical report which includes methodologies for 2006 which was utilised in the 2009 test.</p>
<p>PISA 2006 Technical Report "</p>
<p>Oh very nice. If you click on the link, you will see getting a copy of the technical report costs $92</p>
<p>I take it then, you do NOT have actual evidence that the data are adjusted for poverty levels and inequality. I certainly have seen nothing in the available materials to indicate they were.</p>
<p>‘Yes…I’m sure that PISA spend years compiling data and formulating tests so to provide a comprehensive statistical result, all at the behest of the OECD rich club of nations mind you, because it will be nonsense as it compares apples and oranges.’</p>
<p>The OECD and PISA do not have infinite budgets. They created what they felt would be a USEFUL set of data, not a perfect one. I am not saying they are useless. They SHOULD be used to examine possible improvements in educational policy.</p>
<p>I think that “the sky is falling” stuff is not justified by this data however. I suspect if you asked the folks at PISA, they would agree with me. This data was designed to aid policy makers, not to indict entire societies. Whats being said HERE is over the top. That is why some of us are critiquing the data - its not that its not a worthwhile effort, but its not a strong enough reed to justify some of the extreme statements being made. </p>
<p>You seem to be looking at things black and white - its either perfect or its non sense.</p>
<p>its not non sense, but it also does not reflect every difference between societies. Thats okay, as I am sure PISA expects POLICY makers to take those differences into account. Not every analytic issue has to be reflected in EVERY data product.</p>
<p>Teaching Math to the Talented : Education Next </p>
<p>I looked at that site. Its clearly a very STRONGLY opinionated, site with its own views on educational reform. </p>
<p>I am afraid I cannot trust whether or not its cherry picking data. </p>
<p>unfortunately this debate is not only emotional, its high stakes, with huge amounts of money at stake, and vast potential impacts on millions of peoples lives. </p>
<p>I think a certain humility on all sides is in order. I dont see that.</p>
<p>"Just curious - why isn’t anyone defending the other countries that are ranked similarly to the US? Why isn’t anyone in denial about it? Finland dropped from first place in one of the categories and they’re not making excuses but doing some soul searching to find out what went wrong. Perhaps they recognize that rankings don’t suddenly become horrendously flawed just because their ego is bruised. "</p>
<ol>
<li><p>Most posters here are from the US, and dont particularly care that much about Finland.</p></li>
<li><p>Your use of the word denial already sets up a hostile atmosphere - those who are challenging the implications being drawn from these results do not think they are in “denial”</p></li>
<li><p>In Finland, etc, results like this are, I guess, being used to examine specific possible improvements in education. Not as part of an ideological battle over teachers unions and school reform, not as part of a panic about “decline”, etc, etc. That kind of talk has made it difficult to focus on narrower policy issues. </p></li>
</ol>
<p>I dont see the folks here concerned about these results focusing on specific, reasonable, corrective policies.</p>
<p>MSFHQ is right. This is comparing an apple to twenty bushels of various fruits, vegetables and grains.</p>
<p>Take the student population of any country in the world: Finland, South Korea - you name it and compare their test scores to the top equal number ( 35,000, 250,000, 1 million, etc.) See what you get.</p>
<p>We are a country of 316 million people that works to send everybody to school and tries to send huge numbers to various colleges. There is no where else in the world that does what we do.</p>
<p>You can argue that what we do is not efficient, but we are “testing” massive quantities of students that in other countries of comparable size get funnelled into non-academic programs for which they are not “tested” for these comparisons.</p>
<p>Whether this information is panic-inducing, depressing or simply taken in stride is a personal decision, but I don’t see how anyone can not see that it has negative long-term implications for the U.S. Combine the lackluster performance of American students with the ballooning costs of health care, higher education and possibly energy with respect to the rest of the industrialized world and it does not bode well for our future competitiveness.</p>
<p>China/Shanghai entries this year are not the issue
Ok
For argument sake lets everyone just ignore the China/Shanghai entries on the test.</p>
<p>So now will you say that being ranked 13th 16th & 24th in the OECD is ACCEPTABLE?</p>
<p>
</p>
<p>Where do you get this idea? Plenty of other countries have those same goals, and many are more successful at it.</p>
<p>The U.S. is not a leader in high school graduation rates:</p>
<p>[High</a> School Graduation Rates in Select OECD Countries](<a href=“http://www.data360.org/dsg.aspx?Data_Set_Group_Id=1653]High”>http://www.data360.org/dsg.aspx?Data_Set_Group_Id=1653)</p>
<p>Or in college graduation rates:</p>
<p>[U.S</a>. goes from leading to lagging in young college graduates](<a href=“http://www.washingtonpost.com/wp-dyn/content/article/2010/07/22/AR2010072201250.html]U.S”>http://www.washingtonpost.com/wp-dyn/content/article/2010/07/22/AR2010072201250.html)</p>
<p>BBD</p>
<p>Universities regularly subscribe to a myriad of ‘pay for use’ resources, and make them available via their library’s electronic resources portal. Since these particular PISA methodological documents are used primarily by educational institutions then I suspect printing and sending them to the general public is not a priority for PISA or the OECD, hence the pricing.</p>
<p>If you have access to a University library site with an institutional access password the document should be available. In the UK an ATHENS password allows access to electronic journals, e.books, and digital resources to which the university library financially subscribes. I’m sorry I don’t know what the ATHENS equivalent is in the US but I assume it must exist.</p>
<p>*BBD</p>
<p>Universities regularly subscribe to a myriad of ‘pay for use’ resources, and make them available via their library’s electronic resources portal. Since these particular PISA methodological documents are used primarily by educational institutions then I suspect printing and sending them to the general public is not a priority for PISA or the OECD, hence the pricing.</p>
<p>If you have access to a University library site with an institutional access password the document should be available. In the UK an ATHENS password allows access to electronic journals, e.books, and digital resources to which the university library financially subscribes. I’m sorry I don’t know what the ATHENS equivalent is in the US but I assume it must exist</p>
<p>*</p>
<p>I saw nothing anywhere that gives an indication they adjusted in any way for poverty and income inequality. YOU are making the claim they do. Its up to YOU to show that they do - not merely to site the name of their technical report. You go get the tech report and YOU quote it. </p>
<p>I don’t have to do research to prove a negative. Think about it. Even if I had access to the report, I would have to read the whole thing to show that there is NO such adjustment. You, OTOH, only have to show the part of the report that DOES show the adjustment. Thats why we dont ask people to prove a negative.</p>
<p>*For argument sake lets everyone just ignore the China/Shanghai entries on the test.</p>
<p>So now will you say that being ranked 13th 16th & 24th in the OECD is ACCEPTABLE? *</p>
<p>pretty much in line with Sweden, Germany, France, UK. Though lower on math. </p>
<p>in line with countries with comparable economies, comparable cultures, comparable approaches to education.</p>
<p>I would say it suggests a need for more emphasis in math, but not that the sky is falling.</p>
<p>Clearly this isnt about some unique cultural problem in the USA. Rather there is something unusual happening in east asia. Maybe they have some wonderful approach to education we should all adopt. Maybe they teach in a way that makes kids good at taking standardized tests, and not so good in other ways, as some claim. Maybe they have something unique going on culturally for good and bad, that we cannot replicate, and should not try.</p>
<p>Those are worth thinking about. I don’t think you can tell from the results shown.</p>
<p>On reading, US is ahead of China Taipei (the PC name for Taiwan).</p>
<p>On math and science China Taipei is ahead of US, but still far below China-Shanghai.</p>
<p>What gives? PRC has superior education to Taiwan? Taiwan too westernized in culture? Democracy brings test scores down (but not in Canada?) Also look it relationship of scores for Shanghai vs Hong Kong and Singapore. Shanghai exceeds all others WITHIN the chinese cultural sphere, even countries with MUCH higher levels of economic development. </p>
<p>Note, though, Korea does better than Taiwan, etc. Something is going on here. </p>
<p>And I dont think its what the folks decrying the failing US think is going on.</p>
<p>What to PRC and Korea have in common, not only vs the west, but versus Taiwan, Singapore, etc?</p>
<p>A uniquely successful approach to education? Cram schools and focus on testing?</p>
<p>glido</p>
<p>The UK sends 38% and has a policy goal of 50% of high school graduates to go on an attend university.
Australia sends 32% and the goal is to send 40% of all high school graduates to university</p>
<p>The US sends approx 34%.</p>
<p>we are too concerned about finding our President’s birth certificate to be teaching children, duh</p>
<p>BBD</p>
<p>I know I sound like a broken record at this point but I’m a stubborn sod. :)</p>
<p>…“pretty much in line with Sweden, Germany, France, UK. Though lower on math. In line with countries with comparable economies, comparable cultures, comparable approaches to education.”</p>
<p>The US cannot afford to be be equal with Sweden, Germany, and France.</p>
<p>NOBODY GOES TO SWEDEN FRANCE OR GERMANY FOR A UNIVERSITY EDUCATION.</p>
<p>Swedish students only compete amongst themselves for placements.
German Universities only have a few EU students for a few specific subjects.
France only gets ‘international’ students from French speaking former colonies, Algeria, Francophone Africa and Polynesia</p>
<p>This is not a domestic issue for the US or for the UK
The US and UK are exporters of University education.</p>
<p>Students from Poland, Dubai, Hong Kong, Russia, or Singapore aren’t signing up for
the University of Frankfurt
or the Sorbonne in Paris
or the Swedish University of Agricultural Sciences…right?</p>
<p>They want Oxford, Harvard, Cambridge, MIT, Stanford, Bristol, Manchester, Columbia, USC !</p>
<p>In the UK these test results are seen as a disastrous trend that MUST be addressed.
International Students are an industry…but if UK students cannot compete with their foreign competition, then the UK may just end up running Oxford and Cambridge for international students only.</p>
<p>If US colleges did not allow foreign students then you are right…US High school students could be as dumb as you like…be ranked 55 if you like, who cares? </p>
<p>You think Swedish students care if they are ranked 5th or 50th?
Ain’t nobody coming to take their university slots (who the hell speaks Swedish anyway?)</p>
<p>But American and British students have to compete against the world.</p>
<p>Basically to the OECD this is an external measure, A comparative ranking amongst member countries.
However these rankings mean something very different to the US, the UK, (and Australia).
It is as indicator as to whether they can compete INTERNALLY against other countries since they actively encourage foreign students.</p>
<p>People here arguing that the US system is ‘great’, or that the statistical model might be ‘flawed’, or that its ‘apple and oranges’ are really missing the long term implications of this trend.</p>