Musing

Last year, the College Board and ETS surveyed Readers asking general questions about the reading experience. Hats off to them for listening because this year they provided a hospitality room in the hotel that provided soft drinks, coffee, water, and assorted munchies like chips, cookies, fruit,  and hot pretzels. Then, Saturday night, we had $25.00 added to our stipend for a meal on our own. We’re provided breakfast, lunch, dinner, and snacks at the Ocean Center every day of our reading. Considering the numbers of people being served and their vegan/vegetarian/kosher/lactose intolerant/will eat anything not still moving preferences, the staff and choices are commendable. Yet, honestly, by day three or four, the idea of yet another meal on a divided tray is underwhelming. So, the Saturday night out was a welcome break.

THE SCORING BEGINS

Before serving as an AP Reader, I dutifully downloaded Scoring Guides and dolled out scores ranging from 0-9. What I learned after becoming a Reader was how to more accurately assess an essay based on the holistic definition for that number.

Okay, this still comes off sounding like the Charlie Brown teacher-talk, whaa-whaa-whaa-whaa. Let’s start from the top. . .

When we arrive in Daytona, we look for “the list” which tells us what questions we’ll be scoring; 1, 2, 3 or alternate (overseas, Form B, and makeup exams). Assignments are usually met with groans or glee. I have no idea how questions are assigned. Perhaps that’s an “inner circle” decision or a random-computer generated one.

Some relish the challenge of scoring the synthesis essay; others approach it like punishment for past teacher sins.  I’d hoped to be assigned the synthesis question because the experience of reading for a question helps me to better help my students, but no luck yet. The synthesis question is allotted more readers because they’re sifting through more artifacts. This year, the argument question started out with more readers than we did on the rhetorical question.

In the four years that I’ve scored, I’ve read the argument question twice and the rhetorical question twice. For those of you who remember the infamous “Flamingo” prompt. . . that was my question that year. Wow.

The first day, we report to our Question room and locate our table. New readers are often placed on the right and left of the Table Leader. It’s easier for both because, generally, new readers have more questions. In years past, new readers had different color name tags and a little acorn designating that they were first-year readers. Story is some new readers were offended by being labeled “acorns,” so now only table leaders are differentiated by green name tags. Why would anyone object to more experienced readers reaching out to the first-year people? Nuts. I appreciated being labeled (ahh…there’s the rub…) a first-year person because I could ask truly dumb questions. In fact, my second year I gave serious consideration to exchanging name tags with a first-year person.

At each table, we look for our “place card”‘ which has our name and reader number that we’ll use when we bubble.  We each have a stack of photocopied essays, two sharpened #2 pencils, a white eraser, a scoring guide, and a copy of the prompt. Our Question Two Leader, Larry Scanlon (co-author of The Language of Composition and A Guide to Accompany 50 Essays) starts us off by having everyone at the table introduce themselves. Nothing extraordinarily exciting…name, school, number of years as AP Reader, sometimes courses taught.

Larry reads the prompt to us, then we set about pulling the “benchmark” essays from the stack. The Table Leaders, who arrive at the reading two days prior, pulled an assortment of essays, haggled over their scores and decided which ones fit which scores. Benchmark essays are those that most reflect a particular score. Rangefinder essays are those that reflect low to high within a particular score. So, for example, for a score of 6, I’d have a benchmark 6 with rangefinders of low 6 and a high 6.

To start, Larry will identify certain essays to pull from our stacks (they’re identified by alphabet). So, we might have four essays which represent even-numbered scores. We, individually, read and score the essays. Initially, the Table Leaders will walk around the table asking us how we scored the essays. Then, Larry will ask the room for a show of hands indicating how we scored the essays. Nothing like raising your hand to say you thought the 8 was a 2. . . An exaggeration, but it happens. We’re given time at our tables to discuss the scores and ask questions.

We’ll do this scoring all morning the first day, switching from even to odd to rangefinders, to here’s four essays representing odd and even….all designed to calibrate the scorers. Sometimes, this continues an hour or more after lunch before “live” scoring begins. We’ll do this-though for just a few essays-for almost all the days we score, with the exception of the last two days. Particular attention is given to essays in the 4, 5, and 6 range as these can often be the most challenging to score.

Every morning we’re directed to reread the prompt and the scoring guide. While the first year I thought this tedious, I quickly understood its importance. After the first full day of reading, the prompt itself wasn’t as fresh in my brain, so I’d read a line or two thinking the kid was genius, not realizing it was a line directly lifted from the prompt. They don’t all use quotation marks to indicate they’re using the writer’s exact words!

I also learned that arguing with the Question and/or Table Leader about a particular score is pointless. The benchmark and rangefinder scores are what they are…I can ask questions to help myself understand why the essay was given a certain score, and I’m free to disagree. But consensus rules. Often, once we talk through an essay at the table, I come to understand the decision better.

For at least two days, the Table Leader will “back read” our scored essays. The runners will hand our folders directly to the TL, and s/he will select essays to back read. If necessary, s/he will ask us to re-read an essay if the score is off. The QL can also randomly back read scored essays. Of course, whenever there’s an essay we’re not sure about, we always discuss it with the TL.

Scores of 0 and - must be shown to the TL.  A topic that simply repeats the prompt is a 0. A blank response or one that’s completely off-topic (letters to the readers, sketches, rants).

Stay tuned: Tuesday…Internalizing the scoring guide, what those numbers really mean, and amen.