USA Today and NSSE: project to release benchmark data

Parents Forum
1

<p>The NSSE-USA Today Initiative promises another way for parents and students to navigate that often rocky road to greater transparency and accountability in higher education:</p>

<p>
[quote]
Institutions that participated in NSSE at least once since 2005 are invited to authorize NSSE to provide their most recent NSSE benchmarks scores to USA TODAY for posting on a new USA TODAY college education Web site. The five NSSE benchmarks of effective educational practice for first-year and senior students are: level of academic challenge, active and collaborative learning, student-faculty interaction, enriching educational experiences, and supportive campus environments. Institutions that permit NSSE to release their benchmark scores to USA TODAY will be eligible to be featured in stories that may appear in USA TODAY print publications and on its Web site, including a possible “annual feature” story that coincides with NSSE’s annual report in November. </p>

<p>The NSSE-USA TODAY partnership is a promising experiment in keeping with the accountability and transparency movement in higher education. More information will be posted periodically to the NSSE Web site as it becomes available....</p>

<p>At this point, USA Today is trying to gain a sense of how many schools are willing to respond to the public call for transparency and make plain their institution’s commitment to educational quality by participating. The sooner a college or university commits to releasing their benchmark scores, the more likely it is that USA Today staff will be able to include it for consideration for further investigation....

[/quote]
</p>

<p>Here is a link to the FAQ page of the NSSE-USA initiative:</p>

<p><a href="http://nsse.iub.edu/html/USAT_initiative.cfm#anchor_4%5B/url%5D"&gt;http://nsse.iub.edu/html/USAT_initiative.cfm#anchor_4&lt;/a&gt;&lt;/p>

2

<p>It's hard to argue with making more information available, and NSSE has a lot going for it, but I wonder about the dangers of a list that boils down the results of a large, wide-ranging survey into just a few scores. These benchmark scores, as I understand them, lump together results from only some of the NSSE questions into 5 categories:</p>

<pre><code>* Level of Academic Challenge
* Active and Collaborative Learning
* Student-Faculty Interactions
* Enriching Educational Experiences
* Supportive Environment

</code></pre>

<p>Laudably, the USA Today list would make very clear that this is not a ranking system. (Of course someone will post the results sorted high-to-low by benchmark scores on CC within 45 minutes USA Today's publishing the data ;) )</p>

<p>U. S. News has already made available fuller NSSE data for a number of colleges who have released it. If a school in which you are interested is available, I would recommend having a look at these, which give the full results for many of the survey questions. It can be really illuminating; one school, for example, fell in D's list because the NSSE showed that students there spent the least times doing academic work per week, and hardly had to write any papers, despite the fact that it ranked fairly high in the USNews rankings. </p>

<p><a href="http://www.usnews.com/usnews/edu/college/rankings/ranknsse.php%5B/url%5D"&gt;http://www.usnews.com/usnews/edu/college/rankings/ranknsse.php&lt;/a&gt;&lt;/p>

3

<p>Now schools will start "gaming" the NSSE telling students how important a good rating is to the school's future and the value of their degree.</p>

4

<p>Seems like a good partnership to me despite the potential pitfall that colleges will use USA Today as a vehicle to boost interest, drum up a larger application pool, or feature NSSE data on their website to garner publicity - that, after all, is built into the whole admissions game cycle- colleges need to get the word out and in as neat, quick, and easily digested format as possible in order reach as many up and coming prospective parents and students (consumers) as they can - keeping in mind that just as there are savvy players out there, many are new to this game. Up until now, I find the USA Today education section has put out some fine pieces - the "Cracking the Code of College Admissions" series is a fine example. I would hope USA Today would be able to produce more of this type of article and other accurate and informative feature articles on specific colleges - that said, in the effort toward greater accountability and transparency, parents and students need to know that if a feature article does appear in USA Today in this context it is precisely because that college has agreed to participate in this partnership and has released its data to the USA Today-NSSE joint venture.</p>

<p><a href="http://www.usatoday.com/news/education/2006-11-08-college-acceptance-usat_x.htm%5B/url%5D"&gt;http://www.usatoday.com/news/education/2006-11-08-college-acceptance-usat_x.htm&lt;/a&gt;&lt;/p>

5

<p>asteriskea,
Many thanks for your continued attention to this topic and the links that you have provided. Very helpful and informative as this student survey gets broader distribution and play among the nation's media. Student voices have been neglected for too long and can often give useful insights on the quality of the experience that undergraduates will experience when they get to colleges. This information is highly useful and relevant for college-seeking high school students.</p>

6

<p>Remember when looking for NSSE results that it's unusual for schools to participate every year. NSSE itself recommends a 3-year cycle to its participants. Therefore, if you follow MarathonMan's link and are looking for a particular institution, you may have look through multiple years before finding an institution's data.</p>

7

<p>Updates on the NSSE-USA Today project:</p>

<p>"Searching for Signs of Engagement: How to make NSSE scores work for you"</p>

<p>How</a> to make NSSE scores work for you - USATODAY.com</p>

<p>How</a> to make NSSE scores work for you - USATODAY.com</p>

<p>and</p>

<p>"A Small Step Toward Transparency"</p>

<p>
[quote]
...“Schools that do participate in this initiative will be able to declare and demonstrate their commitment to improving and being accountable for undergraduate education,” organizers of NSSE said in an FAQ at the time they announced the partnership with USA Today. The group’s board believes that “the time has come for participating institutions to stand together in promoting responsible ways to make available information about important, relevant features of institutional and student performance, and to continue to provide leadership for improving the quality of undergraduate education.”...

[/quote]
</p>

<p>Jobs</a>, News and Views for All of Higher Education - Inside Higher Ed :: A Small Step Toward Transparency</p>

8

<p>"Resources for the college search"</p>

<p>Resources</a> for the college search - USATODAY.com</p>

9

<p>The article "Risky Business:Promises and Pitfalls of Institutional Transparency" by George Kuh, that appeared in Change Magazine, is well worth the read for those interested in the thorny issues related to the use of common templates to create readily accessible, "consumer friendly" data bases in the public interest and transparency:</p>

<p>
[quote]
...Be Careful What You Wish For: </p>

<p>While improvement, transparency,and accountability are desirable ends, data can always be misused and misinterpreted. There are two categories of untoward possibilities: the problematic and the unacceptable. Problematic possibilities. One advantage of using the same measures and reporting the results in a common format is to permit institutional comparisons. But simple displays based on institutional averages can be misleading if used by prospective students to “pick the right school.” This is because—as Pascarella and Terenzini (1991, 2005) concluded—individual student performance typically varies much more within institutions than average performance does between institutions. This is true at every level of education...</p>

<p>with practice and patience,we will all get better at deciding what to measure, how to measure it, and using what we learn to improve the quality of the undergraduate experience and other aspects of institutional performance. Efforts such as the Voluntary System of Accountability and similar initiatives comprise a grand experiment. Therefore, above all, we need to try different approaches, do no harm, and refuse to allow pundits to declare winners and losers. There is too much at stake.

[/quote]
</p>

<p><a href="http://nsse.iub.edu/pdf/Kuh%20Risky%20Business%20Change%202007.pdf%5B/url%5D"&gt;http://nsse.iub.edu/pdf/Kuh%20Risky%20Business%20Change%202007.pdf&lt;/a&gt;&lt;/p>

10

<p>This quote from the Kuh article that asteriskea quoted above says it all:</p>

<p>
[quote]
individual student performance typically varies much more within institutions than average performance does between institutions.

[/quote]
</p>

<p>When you collapse all the NSSE questions down into 5 benchmark categories, it flattens out any meaningful distinctions, making it hard to draw any conclusions except in the very rare cases where a college is a real outlier. Almost all institutions are within a couple of percentage points of the average for their institution type on all of the measures, and the scores for schools that you'd consider peer institutions cluster even more tightly. </p>

<p>To their credit, the USA Today article is careful to warn that the differences of a couple of points are not meaningful, and suggests a 5-point difference as a good indicator of real disparity. (If only USNews were as honest about the limits of its methodology ;) ) It's hard to find that sort of difference between two colleges that you'd think of in the same sentence.</p>

11

<p>I can't get the search feature to work on this site -- the link to the searchable database appears to be broken -- anyone else have problems?</p>

12

<p>A short followup to this older thread: it looks like the US News site has been updated recently with collated data from 2007 NSSE reports (data from 2006). These reports are the source for the USA today benchmark figures (see the link in in post #7). Few selective colleges release NSSE figures, but this year's reports include Emory, Middlebury, and Harvey Mudd, among others. </p>

<p>USNews.com:</a> America's Best Colleges 2008: National Survey of Student Engagement</p>

<p>A must-read if any schools of interest appear on this year's list.</p>

13

<p>These survey results of student engagement are at best suggestive, but not measures of anything. They have the usual problem of all voluntary response surveys: it's not valid to say that they necessarily reflect reality. </p>

<p>One professor of statistics, who is a co-author of a highly regarded AP statistics textbook, has tried to popularize the phrase that "voluntary response data are worthless" to go along with the phrase "correlation does not imply causation." Other statistics teachers are gradually picking up this phrase.</p>

<p>


</p>

<p>Math</a> Forum Discussions</p>