I am trying to find that article. It was on the Collegeboard website…I think under " the Professional" part of it. It described the grueling process of how a question finally appears on a real SAT.
If anyone knows how to find that or has it, please let me know!
I think it used to be here:
https://professionals.collegeboard.com/higher-ed/recruitment/sat-test/dev
But they have replaced it with this:
https://collegereadiness.collegeboard.org/educators/higher-ed/test-design/test-development
The “wayback machine” [a name I love] has an archived copy here:
Hope that helps…
Thanks! But the one I wanted goes into huge detail.
I saw what you referenced but the article I was thinking of went into great detail about how they write a question. Then they review it among professionals. Then they put it in an experimental section to see how many students got it correct or missed it. They also see if there is any cultural or political bias. Then they re-write the question, test it again, etc. Then after the question has been tweaked and tested at least 4 times, the question finally appears in a real test.
I think it’s interesting that they have replaced that article since ETS is no longer writing their questions and Collegeboard is now writing them. I believe ETS wrote the question from the SAT for over 50 years and i find it very disconcerting that the questions are now written in house. I do not the quality of the questions will be nearly as good as they used to be…
This a very (VERY) old article, several paragraphs of which describe the ETS’s test development stages, starting with this segment:
http://www.theatlantic.com/past/docs/issues/95sep/ets/fall.htm
You want information about item development in test development. I think they have that on the Research pages of the College Board.
Looks like CB is developing a new question authoring platform called “Hive”.
https://www.collegeboard.org/about/careers/job-listings
"The College Board, a national educational organization, is conducting a search for a Senior Requirements Analyst in our IT Department, who will be resident in our Reston Virginia office.
We are building The College Board’s next-generation item authoring platform and assessment development platform, called Hive. Questions on the SAT, PSAT, and Accuplacer assessments are meticulously written, re-written, measured, tested, and balanced before they are ever presented to students. Hive will manage these activities, in addition to providing versioning, workflow, and test assembly features. This is a mission critical application for The College Board. Hive is also a brand new capability for us, which provides us a large degree of freedom to experiment and evolve our software development and customer delivery processes. "
Here is the job description:
“Work alongside other members of the testing team in helping develop test strategies and test plans by utilizing creative approaches using mind mapping tools and techniques.
Assist with development tasks, coding automated test scripts to enhance testing efficiencies, and providing business/requirements analysis assistance as needed
Report and communicate testing progress to lead test engineers, project managers, and business partners
Create test automation within assigned projects to increase return on investment by increasing testing efficiencies
Work closely to facilitate and streamline testing with other disciplines (UAT, Accessibility, Security, etc.).
Leverage Jira to capture, manage, track, and report defects.
Leverage test case management tools (Zehphyr, Test Rail) to create, execute, report, and manage testing scope and status.
Create and maintain web test automation framework(s) based on Selenium, that can be leveraged across multiple projects. Train and mentor other team members in regards to utilizing and updating the framework(s).
Create, maintain, and own the test automation strategy. Create ‘best practices’ and guidelines on automation criteria with regards to what, when, and how.”
So is “test automation” automation of authoring or automation of test delivery (i.e., test will be delivered via computer and not on paper)?
I remember reading something like that but my googling skills are not strong enough to find it now.
It doesn’t look like it’s relevant any more though. The College Board seems intent on aping the ACT’s writing-solidly-constructed-standardized-test-questions-is-hard-let’s-just-increase-time-pressure-instead strategy.