<p>In the past, I have posted the guidelines used by Pearson(s) in the training of their "experts.**" </p>
<p>Here's a pretty similar list:</p>
<p>Scoring Criteria*</p>
<p>1) Read quickly for an impression of the whole paper and score immediately; do not reread or analyze.
2) Read the entire paper; the writing sometimes improves dramatically as the writer goes on.
3) Read supportively; look for and reward what has been done well, rather than what has been done poorly or been omitted.
4) Take everything in the paper into account: organization, development, spelling, diction, sentence structure -- everything.
5) Try to ignore the quality of handwriting.
6) Do not judge a paper by its length; some short papers are good, and some long papers are poor.
7) Remember that use of a literary example does not make the paper inherently better than a paper that uses examples from personal experience, history, science, politics, sports, etc.
8) Use the full scale; since papers are ranked against each other and not against an ideal, some papers may receive a score of 6.
9) Remember that each score category represents a range (e.g., a high 3, a solid 3, and a low 3).
10) Remember that each paper is essentially a first draft written under test conditions in only 25 minutes.
11) Remember that an unfinished (but developed) paper is not penalized for lacking a full conclusion.
12) Remember that, even though their writing is being evaluated for readiness to handle college-level work, the writers of these papers are, for the most part, only 16 or 17 years old.
13) Remember that a paper with a score of 6 need not be a polished essay.
14) Remember that any paper that addresses the essay assignment should be scored. The paper receives no penalty for a tangential approach to the issue; it is scored on its own merits and on the logic of the argument developed in response to the essay assignment as the writer interprets it. However, an essay not written on the essay assignment will receive a score of zero.
15) Remember that the standards are set by consensus and that individual readers are expected to accept and to follow those standards.
16) Remember that to the writer of the paper, the ideas expressed are
fresh and original. (Unlike the readers, the writer has not already read dozens of papers on the particular topic.) </p>
<p>** The quotations around the term "expert" relates to Pearson's decision to mainly recruit from a pool of candidates that, at best, had not be known to be great judges of writing talent, let alone able to develop it through teaching. Fwiw, I still believe that the only reason a scoring machine was not used was to pretend the scoring method might account for the type of individual nuances and variances humans are known for.</p>