UO researchers developing a diagnostic reading assessment

Sarah Carlson (left) and Gina Biancarosa
Sarah Carlson (left) and Gina Biancarosa

A new reading comprehension test for elementary school students being developed at the University of Oregonshould help teachers diagnose why youngsters wrestle with comprehension and could inspire new interventions.

The tool is the Multiple-choice Online Cloze Comprehension Assessment, or MOCCA. Leading the way are Sarah Carlson, a research associate in the UO's Center on Teaching & Learning, and center associate Gina Biancarosa, also an associate professor in the Department of Educational Methodology, Policy, and Leadership in the College of Education.

Standardized tests, including the UO-created DIBELS and easyCBM, are deployed in schools nationally each fall as screening and benchmark measures. They identify at-risk children who are struggling readers, but when the problem is comprehension, existing approaches don't go far enough, Biancarosa said.

"There are students who, despite having all of the component skills that are required for good comprehension in place, don't comprehend at a level we expect," she said. "We don't have any tools that enable us to understand why. Teachers are not getting useable information other than a child is not comprehending well."

The MOCCA consists of seven-sentence stories with the next-to-last sentence omitted in each. Students infer the missing sentence by choosing the best of four multiple-choice options.

"MOCCA helps identify the types of thinking processes that readers use, and, more importantly, it points to processing differences tied to good, average and poor comprehension," said Carlson, who recently received funding under a U.S. Department of Education postdoctoral fellowship.

MOCCA is built around understanding narrative and so far the research suggests that many readers who struggle with comprehension are paraphrasers or lateral connectors.

"Paraphrasers tend to work within the goal of the story," Biancarosa said. So, in a story about Sarah trying to get a dog, "they might say, 'Oh, Sarah wants a dog.'” Lateral connectors might say, 'Oh, I want a dog, too.' Lateral connectors tend to make elaborations and relate what they've read to something in their own life.

During the summer, Biancarosa, Carlson and Ben Seipel, an assistant professor of the School of Education at California State University at Chico, began revision and enhancement work on MOCCA. They have written over 400 new items.

They also are converting it from a paper-and-pencil assessment into a computerized version. Experts from Leiden University in The Netherlands, the University of Minnesota at Twin Cities, University of Illinois at Chicago, Ohio State University and the Eugene-based Oregon Research Institute also have provided guidance.

MOCCA launches its first wide-scale field test in spring 2015 among third- through fifth-graders in diversely populated schools in the Eugene, Springfield and South Lane school districts and in Chico-area schools. Teachers from the Eugene-Springfield schools also will help review the items for grade-level appropriateness and cultural relevance.

MOCCA was born at the University of Minnesota, where Carlson and Seipel were doctoral students working under a U.S. Department of Education pre-doctoral fellowship with Kristen McMaster, a professor of educational psychology.

A technology administration agreement shares the rights of MOCCA between the UO and University of Minnesota. MOCCA's enhancement is being done under a $1.6 million, three-year Department of Education grant managed through the UO.

In addition to the spring pilot, Carlson and Seipel are holding "think-aloud sessions" with students in Eugene-Springfield, South Lane and Chico elementary schools to gather more data. "We are asking participants to read aloud and pause at different points to tell us what they are thinking to get a refined sense of what's going on in the reading process," Carlson said.

At the end of next spring's pilot run, Mark L. Davison, professor of educational psychology of the University of Minnesota, will analyze how the test items behaved. His work, Carlson said, should help to identify types of poor comprehenders and further refine the test.

In the grant's next two years, MOCCA will advance to nationwide sampling for comparison with other new experimental diagnostic tools being tested and additional refinements.

"We hope that our tool helps in the development of new or building onto previous interventions," Carlson said. "Some of our colleagues who have helped us develop MOCCA are involved in interventions. This might fit well with their work and inform more focused interventions."

—By Jim Barlow, Public Affairs Communications