by Neil Chyten
No one wants to believe that they are being manipulated. HOWEVER, when it comes to wrong answers on the SAT, responses that you think are careless are actually very carefully designed products of the testing companies’ evil genius. And they pay lots of money for your pain!
I have been in the test prep industry for more than three decades. During that time, I have heard thousands of students say thousands of times that we should not worry because their wrong answers are just “careless errors.” In fact, one such message came to me in an email earlier this morning prompting me to write this article. Of course, the implication is that these students won’t make the same kinds of errors on the actual test. Most students believe that their careless errors are just “stupid mistakes” made because they were tired, they were not focusing, or they just weren’t taking the practice test seriously enough. The truth is, however, that most of these errors are not careless at all; in fact, they are well planned by the brilliant test construction teams of American College Testing (ACT) and the College Board (SAT).
Take it from someone who has written thousands of practice questions for his test preparation books: test- question construction is a precise art form that is designed to elicit a very specific set of responses from students. When I write questions and answers for SAT-like Reading passages for my books and practice tests, for example, I follow the lead of the College Board and ACT by writing answers that will attract a certain percentage of unwary students who are not fully versed in the rules of test construction. More specifically, they are not paying close enough attention to the relationship that exists between 1) the source text, 2) the questions, and 3) the answers. As with all three-headed monsters, you can’t defeat this one unless you understand its anatomy.
Here’s an SAT-type Reading Test example I wrote for the TASCA Test that compares students’ ACT and SAT test taking ability. The source text is an adaptation of a speech made by American president Franklin D. Roosevelt in 1936.
“Of course we will continue to seek to improve working conditions for the workers of America—to reduce hours over-long, to increase wages that spell starvation, to end the labor of children, to wipe out sweatshops. Of course we will continue every effort to end monopoly in business, to support collective bargaining, to stop unfair competition, to abolish dishonorable trade practices.”
Q: Roosevelt’s statements about working conditions in America imply that
A) many American workers have been unfairly exploited by their employers.
B) children have been forced to work long hours for low wages.
C) illegal business collaborations have resulted in lower wages and worsening working conditions.
D) we cannot continue to produce subpar goods that if we are to compete in world markets.
You will notice that the subject of each answer is touched upon in the source text—yet only one is correct. That is because in only one answer is every single word correct. Can you guess the correct answer? Most of you will, but a certain percentage of you will not; and that is exactly the intention. SAT items that consist of a source text, a question, and four answers are designed to make a certain percentage of students answer them incorrectly. The items are written, discuss, tweaked, pretested, tested, tweaked again, categorized, and finally included in a live SAT. Taking that one test item from design stage to the big stage costs the College Board over $1000. But by putting each of the SAT’s 154 questions through this rigorous process, the College Board is able to control outcomes on both a micro and macro level.
On a micro level, they know that perhaps 20% of students will guess B) because the passage does mention “children” and the statement itself is likely to be accurate even though it is not supported in the passage. Perhaps 5% of students will guess C) and another 5% will guess D). In fact, these outcomes are intelligently predicted based on the results of pretesting. The correct answer is A). The word “imply” in the question is important because it indicates that the correct answer will not be directly stated. In such a case, students must identify incorrect words in the incorrect answer choices. Again on a micro level, each test question is a microcosm of the test itself. Like the test, each question requires a great deal of focus and concentration. By placing a time limit on each section, the College Board increases test-taking pressure which, in turn, makes students much more likely to select comfortable answers that merely sounds correct but which are not supported in the passage.
On a macro level, the sum of 154 predictable questions makes the SAT a predictable test. It is predictable by both the College Board psychometricians and by savvy test preparation organizations such as NC Global Education. Because I have spent thousands of hours breaking down tests question by question, I can easily identify, and I can teach students to easily identify, incorrect answers in order to avoid “careless errors.” Our ability to identify incorrect and correct answers is not just true of SAT Reading questions; it is true of SAT Writing & Language questions and SAT Math questions as well.
So, understand that the next time your child tells you not to worry because all of his mistakes are “careless errors,” you should beworried. Your student has fallen into the same trap that millions of students have since the SAT came into existence 90 years ago. Or perhaps I should say, “Don’t worry but let us help! Remember, careless errors are not careless at all; they are actually very carefully designed products of the testing companies’ evil genius.
President and Founder
NC Global Education, Inc.