Who is Writing the Questions for the Khan Academy Redesigned SAT Prep?

Test Preparation SAT Preparation
81

I am still working my way through the 4 Redesigned Practice SAT’s, but my impression that the official practice tests are different from the Khan Academy practice is getting stronger.

Consider these:

http://i.imgur.com/Hdq4Ptd.png

The last two math questions of Test 3, Section 4 (the last two calculator grid-ins) have very, very long complicated wording and very, very easy numbers. They are definitely different from the hard questions on the current SAT.

I think these kinds of questions will give some students trouble. But I haven’t seen anything on Khan at all similar that might prepare students for such questions. In fact, the difficult Khan questions are hard in a completely different way.

I am no expert on Common Core, but these questions don’t look to me like school questions either.

82

Wordy, yes, but not ambiguous or unusually difficult. English language learners might struggle with these the most.

The definition of “school questions” can vary widely, but I’m a little unsure how this is a “real life” question…

83

I am not an English language learner, but I found the formulation of the question very confusing and its presentation illogical.

We are told that the general formula can be applied to any “part” of the store, such as a department or a checkout line.

This formulation stopped me because I thought it was strange to call both a department and a checkout line a “part” of the store. To me a line is a group of people standing in order. The line is in the store but not part of the store. I would have called it the checkout “area”.

I later googled it and indeed, Little’s law is for “queueing systems”, not for stores as such. This explains that the correct general concept is a “system”,not a store, and the extensions are to “subsystems”, not to physical or spatial parts. This makes much more sense.

In addttion, the language was so long and repetitive that I was expecting a trap, that maybe the formula would not apply.
But then it turned out to be just a lot of excess verbiage. There was more reading than in the reading part. Three paragraphs to get to one elementary question.

84

This thread has been inactive for a couple of months, but I would like to reopen it to compare opinions on the verbal part of the Khan Academy SAT prep.

I have been working through the verbal material using the 4-step practice suggestions that appear across the top of the page: Reading (passage type), Writing (type); Grammar (type); Timed-Mini Section (13 minutes). I haven’t done any of the additional practice listed lower down. The first set I completed was at level three, and all the subsequent sets were at level four.
Here are some observations:

  1. The quality of the material in the Reading, Writing, and the Timed-Mini sections is higher than is the quality of the math questions I looked at a couple of months back. However, it looks to me as though there are some slip-ups.
  2. Some of the answer choices seem more ambiguous than are the choices on the official practice tests. This makes me wonder whether this will carry over to the real test, or whether this is a just an artifact of poor writing and/or editing on the part of the Khan Academy team. Because of this ambiguity, in some cases I had to rush to finish the timed sections within the allotted time. Several times I had to go over the passage again and again combing the text for the justifying lines that were not there. Maybe I am using the wrong approach, but I find this a little surprising given that I usually can complete a reading section in less than half the allotted time.
  3. The writing of the reading passages themselves is sometimes not very good. I even found a number error in an original passage – something to the effect that some people using Facebook have more friends, some have less friends (instead of fewer). I imagine this error really is in the original, but shouldn’t this be edited before it is put into a test?
  4. The quality of the Grammar questions (the third category) is low and not at all SAT-like. They are poorly constructed drills. These seem to have been written by a completely different team than the team who wrote parts 1,2 and 4.
  5. The timed mini-sections in part 4 are similar to the reading passages in part 1, with the addition of graphs and questions about the graphs. I have done quite a few of these, but I have never had a timed section with writing. Where is the timed writing practice? Am I missing something?

Here is an example of a question that bothered me. This is not the worst example I found, but it is the first one I bothered to copy.

http://im***gur.com/share/a/G9FjR

This is a double passage. The first question I posted asks what the author of passage 1 says about the difference between present-day America and America of the past.

Examining the passage, the main differences stated by the author of passage 1 are:
a) present-day America has a much higher population
b) present-day America is made (racially) integrated by the following:
i) transportation
ii) a common language
iii) literacy
iv) accommodations laws in some states
v) fair employment laws in some states

Based on the passage, I would expect the correct answer to be that present-day American is more racially integrated.

The answer choices are: A. more spiritual B. greater equality of opportunity C. education in higher esteem D. less interest in traditional social values.

A, C,D are clearly wrong. Is B correct? While the main emphasis in the passage is on integration, there is clearly some evidence that there could be less inequality of opportunity in 1962 America versus the America of the founding fathers. For example, fair employment laws should facilitate access to jobs, and this in turn should create more opportunity. However, this answer choice does not focus on the MAIN difference stressed by the author, which is integration, not equality, especially integration of public transportation. Indeed, in 1962 there was a huge amount of racial inequality of opportunity, and there was not so much integration outside of the public transportation system, and this was a large part of what the civil rights movement was about. No senator arguing for civil rights was going to use the greater equality of opportunity as a REASON for civil rights legislation. The REASON for civil rights legislation was the INEQUALITY of opportunity, despite the racial integration of public transportation. Even after Brown versus the Board of Ed, many school systems remained segregated because neighborhoods were segregated.

The second question asks for the line numbers that justify the answer to the first question. After having settled on “greater equality of opportunity” as the answer to the previous question, I looked for the answer choice with the line numbers about fair employment laws. These line numbers were not among the choices. Then I went back and went through the line numbers of the answer choices one by one.
A: “Nothing is eternal except change” – not related to equal opportunity.
B: “Since the accommodations act, America has changed” – does not specify how America has changed.
D: speech of Lincoln – not related to present-day America
C: “America is integrated. Public transportation makes America integrated.” THIS is equality of opportunity???

Clearly C is not as bad as A,B, and D. But integration of public transportation is not the same as equal opportunity. Equal opportunity is not about equal access to seats at the front of the bus. In the first instance, equal opportunity is about equal access to employment opportunities, and in the second instance to wealth. education, power, etc. So the correct answer choice would contain the line numbers about fair employment laws.

I checked the Khan Academy explanation of the answer choice. It reads “This is the best answer choice. The author states that America is an integrated nation, made this way by public transportation and language.”

If the writer of this explanation is equating the integration of public transportation with equal opportunity, then he or she has not understood much about the civil rights movement. America in 1962, or EO law. This is not good for someone writing official SAT American history questions.

85

@Plotinus Agreed, I looked at the Khan Material a few months ago (particularly Reading) and gave up in frustration. I’ll copy paste what I wrote on another website about this:

Any thoughts on the quality of the practice material released by KA so far?

I’ve worked through much of the reading so far & have had mixed feelings. The material seems quite uneven in quality. It’s always been the case that CR is hard to simulate, and I wonder how much the CB is actually guiding KA in development.

The two part questions (provide the lines to support the previous question) have been vexing. I’m still trying to figure out the “standard of evidence” that would indicate a given line supports a given answer, but sometime it’s tricky. This coming from someone who typically runs through old SAT reading without breaking a sweat.

There have also been some pretty head-scratching errors that have led me to doubt the other questions. Here’s an example:

In this image, you will see an answer to a question with an explanation that cites a portion of a passage including the phrase “courtesy due” (lines 29 in the text).

In the next question, you have to find the lines to support that previous answer, but lines 27-31, from which “courtesy due” is excerpted, are marked as incorrect!

Confusing :frowning: I’ve reported this and other errors to KA. It’s obviously still early, so hopefully they sort out the major hiccups. Anyone else have any experiences with the new material?

86

@JBIG1994 - your links are broken/blocked by CC, I’m afraid.

87

@JBIG1994 Thanks for sharing your impressions. At this point, my concern is not so much about the errors and questionable quality of the Khan Academy prep. My concern is more about whether the revised SAT CR will be different from the old SAT CR is ways in which the Khan Academy prep is different. Some of the differences I have noted are:

  1. It is much more difficult to answer questions without reading through the entire passage first. For the old SAT, I advised slower readers to answer line-number questions after finishing those lines. I found this method does not work well at all with the Khan Academy prep.
  2. Questions and/or answer choices often do not focus on the main idea of a passage or paragraph. For the old SAT, grasping the main idea was usually the key to answering questions.
  3. It is much more difficult to anticipate the answer based on the passage. Many questions have to be back-solved. The need to evaluate answer choices one by one makes the process more time-consuming.
  4. The “correct” answer is sometimes not so clear. For the old SAT, one answer choice was clearly right and all the others were clearly wrong. In the Khan Academy prep, there are questions in which all the choices are bad, but one is less bad, or in which more than one answer choice is reasonable. I know students sometimes claim that two answer choices are correct on the old SAT, but I have never found this to be the case. One of the two answer choices always has a clear mistake in it (sometimes just one word).
88

^ These issues that you have raised about the Khan Academy reading prep – do you also see them in the 4 released practice tests? Or just on the additional material available at K.A. ? I don’t know for sure why I think this, but I assumed that the 4 tests were developed internally by the College Board and the rest was farmed out to KA.

89

@marvin100 Sorry about that, just go to “imgur” "dot com) and then append the rest of the link.

@Plotinus I’ve found much the same as you have with the Reading sections. I VERY rarely make a mistake on old SAT Reading (we’re talking 1/100 wrong and that’s just usually because I’m racing through the test quickly to get it over with) but on KA I would sometimes get 2-4 questions wrong per set of 10! I chalked it up at first to the “learning curve” of understanding a new test’s style, but over time I realized that it wasn’t that simple, and that practice quality had to be part of the issue.

90

@pckeller I have not worked through all the reading sections of all four tests, but the ones I have completed do not have the same issues as the KA reading practice. The most notable difference is in the “what lines justify your answer in the previous question” questions. I was able to anticipate/forward-solve these for the official practice tests; I often had to backsolve these for the KA practice (and even after backsolving sometimes was scratching my head or fuming).
I find this difference significant because my standard procedure is to tell my students to look for the answer in the passage, not in the answer choices. If the real test is going to be like the Khan Academy practice, there will be strategy and timing consequences.

@JBIG Yes, I blush to admit that when I first started doing the KA practice, I too was impatient to get through the practices quickly and got some wrong. I have found that by carefully backsolving if I have uncertainties, I have been able to find the correct or least bad answers. Some questions seem to me just defective.

The result is that during the timed 13-minute sections, I don’t have such a big buffer of remaining time. However, the last three questions are usually on the graph, and these questions take much less time to answer. So it is fine to go into these three questions with only 2-3 minutes left.

The timing on the real test is completely different because there is one big block of time for all the passages. Students who can concentrate well for a long time will like this better; students with short attention spans are going to have issues. In any case, there is very little on KA to help students optimize their timing for the real test format. There really should be a large set of 4-passage 65-minute reading quizzes. The average digital native is not going to make it.

91

It has always been hard to convince students not to waste prep time on fake materials. But this time around, it is even harder. First of all, there are so few real materials available. And then you have KA being designated by the college board as the official supplier of SAT practice. But all of the same reasons for avoiding pseudo-tests still apply!

For now, I am going to encourage my students to stick with the “official” 4 released tests and just analyze them to pieces. Learn those 4 tests until you could teach someone how to answer any of the questions…I am not convinced that the next best material is better than nothing!

92

I would not say that the Khan Academy prep is worse than nothing – it could be helpful in some ways to some students. However, I definitely think that the Khan Academy prep is worse than the old Official SAT Online Course. I thought Khan Academy was going to give the Online Course an interactive update with video explanations and personalized study programs using official practice material (where “official”=“written by the same people who write the test”.) It doesn’t take genius to figure out that Online Course+video explanations+automated diagnostic tests and study plans = a good, up-to-date online prep resource. Instead, Khan Academy= videos+automated diagnostics+imitation practice material. How close is the imitation material to real material? We still don’t know for sure.

If CB is “partnering” with Khan Academy, why didn’t College Board have the ETS people produce the questions for Khan Academy, the way CB did for the Official SAT Online Course? Why was the most inimitable element of all – writing the questions – sub-contracted to people who are clearly not competent? Was it just to save money? Or did CB not want the prep to be “too good”?

93

My guess is that it just costs so much money to produce quality material that they simply can’t (or won’t) give it away for free. The CB has never given out a ton of free practice SAT questions - it’s all been behind a “paywall” of sorts, so perhaps they think it’s fine just to let KA give it their best shot if it’s going to save them (perhaps) millions of dollars spent constructing the questions, evaluating them, testing them in the field, etc. etc.

94

The Official SAT Online Course cost $70 for a year’s subscription, and $60 if you had purchased a copy of the Official Guide. I don’t know how much money it cost CB to develop the Online course material, but surely some of it could have been raised from the same foundations that are funding KA. In any case, it was a very modest price for students to pay – about the same as one SAT test sitting. CB could have charged $60 for those who can pay and a fee waiver for the same people who get fee waivers for the test. If a course with official practice material were available, I might encourage more students to stick with the SAT instead of switching to the ACT. As it is, I am moving more and more students who need a lot of practice to the ACT. The decision to contract out the question-writing was a very bad, short-sighted decision all around.

95

Anyone know whether the “Daily Practice for the New SAT” app, advertising “hundreds of new official questions”, is using Khan Academy questions?

96

@bctnin1059
I don’t know who is writing these questions, but they do not seem to me like Khan Academy questions.
The difficulty level is not very high.
I worked through a bunch of the level 3 questions (the hardest available) and found that well over 50% of the people answering the questions answered them correctly. Indeed, they were quite easy, around a level 3 out of 5 on the old scale.

97

@Plotinus the highest difficulty level in Khan is level four. Originally I thought that you if the little arch thingie was complete that meant the student had completed that section but it means they are just begin level four and you have to look at the fine print to see how many the have to complete to finish level the level “Level Four questions completed: 4 or of 44”

DS18 is working through level four and said it’s not so much the math but figuring out exactly what they are asking. One question dealt with insurance deductibles, DS has zero knowledge about insurance but he was able to figure out what they were asking by reading the explanations. I think it those kind of questions that require knowledge outside of the actual math that are going to cause kids problems, especially with kids that don’t prep.

98

@3scoutsmom
In my previous post I was addressing the question by @bctnln1059 about the Daily Practice for the New SAT, not the Khan Academy practice.
The Daily Practice has only three difficulty levels, and the questions I did at the highest difficulty level seemed relatively easy but more similar to CB/ETS material in style and feel than are the Khan Academy questions. This is just my subjective impression. I don’t know who is writing the Daily Practice material.

99

@Plotinus Thanks for the clarification! So are the Daily Practice questions for the new SAT or the Old one? Though if they are as easy as you say I might not bother suggesting them to my kids.

100

@3scoutsmom
The Daily Practice Questions are for the redesigned SAT. Although they seem easy to me, I think they are worth doing. My subjective impression is that they have more of the look and feel of CB questions than do Khan Academy questions. The reading passages give a sense of the kind of passages that might turn up on the real test, maybe with harder questions. The passages are definitely different from the passages typical of the old SAT. However, the archive of questions is very small.