Standardized Tests: The Movable Target?

A couple of weeks ago, a Washington Post article stated, “Reading scores for the Virginia Standards of Learning test dropped by double digits following the introduction of a new, more rigorous exam this past year.

A couple of years’ ago, it was Virginia’s math scores that dropped as a result of newer, more rigorous standards.

Meanwhile D.C. Public School students made impressive gains in math and reading tests, but math officials reported that the gains were in part tied to the division’s decision to “score the tests in a way that yielded higher scores even though D.C. students got fewer math questions correct than in the year before.”

Virginia and D.C. are by no means alone as they try to balance tougher standards with consistent scoring. In 2012, seeing scores plummet, Florida state education officials decided to lower the passing cut rate/grade. A couple of months ago, the debate played out again with the Florida State Board of Education again voting to prevent school grades from dropping more than one letter grade. Even Board Member Kathleen Shanahan, who was a driving force behind the school grading system that “served as a model for other states,” commented, “I am struggling with the integrity of the accountability system…and the reliability of grades.”

As the debate plays out amongst politicians and state boards of education, it is teachers and students who are caught in the middle.

Should standardized tests enable educators to compare student performance from year to year? Yes.

On the other hand, standardized tests should challenge all students. But, at what expense? In Virginia, the more rigorous tests and scoring resulted in significantly lower student scores. State officials talked up the results as a temporary price to pay as Virginia shifted to higher standards.

But who paid the price? Students. Students, who in essence served as guinea pigs, and as a result may not earn and Advanced Diploma or even a Standard Diploma.   

Like too many classroom grading systems, proficiency on state tests seems immeasurable and perpetually alterable based on the whims of a few elected or appointed officials. Instead of being based on teacher instruction and student learning, passing rates and standardized testing have become politicized (the consequences range from the grade the school receives, to funding, to accreditation,  to employment decisions, to student graduation) while failing to galvanize meaningful educational reform.

We cannot expect our teachers and students to hit a moving target.

What Works In Education: How Educators Can Make A Difference

Through a meta-analysis of over 900 studies, John Hattie’s Visible Learning evaluated the impact of many factors on student achievement. Using “effect size”, Hattie ranks factors from family structure, teaching practices, socio-economics, school policies, and more on student learning.

An effect greater than .40 is seen as above the norm and leading towards greater-than-expected growth. An effect size of .40 is the expected progress a student should make within a year. An effect size of .6 means that the relationship between one factor and student achievement was 60% of one standard deviation.

Effect size = Average (post test)- Average (pre-test)

                              Spread (standard deviation)    

While schools only have our students for 7 hours a day, Hattie’s meta-analysis proves teachers have an amazing ability to influence learning—and even overcome factors beyond teacher control.

Influence	ES
Student expectations	1.44
Formative assessments	.90
Feedback	0.75
Teacher clarity	0.75
Teacher-student relationships	0.72
Creativity programs	0.65
Professional development	0.62
Not labeling students	.61
Concept mapping	0.6
Mastery Learning	0.58
Student-centered teaching	.54
Full vs pre-term birth weight	.53
Socioeconomic status	0.52
Availability of resources at home	0.5
Parent involvement and achievement	0.49
Overall teacher effects	0.32
Ethnicity	0.32
Homework	0.29
Family Structure	0.18
Ability grouping	0.12
Male-Female Differences	0.12

Red: influences beyond teacher/school control

We cannot continue to make excuses. There is no place in our schools for teachers who speak fatally: “His family life is horrible; he’s not going to be successful” or “My students aren’t motivated” or “He’s dyslexic”. Such cynicism has no place in our schools.  We need teachers who are up to the challenge and believe in their students and their own abilities.

While factors beyond our control influence student achievement, expert teachers focus on the 45 factors that have a greater influence than socio-economic status.  Of these, teachers and schools control and influence all but 2 (birth weight and home environment).    

Good teachers believe in their abilities. They maximize their impact by using research-proven methods. They know they can will make a difference. They don’t label their students (.61 effect size).  Good teachers don’t use a student’s disability, background, ethnicity, family, etc. as an excuse. Instead they embrace a growth mindset, a belief that good teachers make a tremendous difference.

We make a difference.

Sadly, Hattie’s research also illuminated how many teachers, schools, and policy makers continue to support policies, programs and interventions that don’t work, or actually negatively impact student learning.

Only influences above .4 are considered above the norm, leading to more-than-expected growth.

Some examples of educational movements that have low—and in the case of retention, negative—impacts:

Charter schools: .20

Gender-based instruction: .12

Whole language learning: .06

Open classrooms: .01

Multiple intelligences/Learning styles: .17

Retention: -13

If we are to move education forward, we must fully embrace research. We must confront what really matters. We can no longer make decisions based on our beliefs, anecdotal “evidence” or “that’s the way it’s always been.” The past two decades have seen an explosion of educational research, which we must embrace. No excuse exists for educational leaders continuing to make ill-fated decisions based on instinct or personal convictions.

Let’s ask ourselves: What can we do better? How can we progress? How can we make learning visible to the teachers?

________

Hattie, John. Visible Learning for Teachers: Maximizing Impact on Learning. London: Routledge, 2012. Print.

Grading Group Work in a Standards-Based Classroom

The other day, Ms. Snider approached me with a simple query, “Do you have a minute to discuss grading of my group project?”

The ensuing conversation centered around two questions:

1.     Should the group be assigned one grade?

2.     How can we assess for individual learning and provide meaningful feedback to each individual?

We began with a resounding “NO!” to the first question.

Both of us believe that a grade stands for what each student learns, so this was easy. But, as we explored the subject in more detail, we soon realized that assigning the group one grade actually defeats the purpose of a cooperative assignment. Ms. Snider shared a story of her daughter, a high-achieving student, who often felt pressured to pick up her group mates slack. Like many students, her daughter felt it easier to do their work than to wait for them or to work with them.

Is it any surprise that a lot of our top students, cringe at the thought of group work?

We had our starting point: individual students will be responsible for their own learning and will receive their own grades.

Our next challenge was to determine how students would be held individually accountable while still relying on each other to successfully complete the assignment. Although this was a new project for Ms. Snider, she knew exactly what she wanted the students to learn and what skills they would acquire. That led us to our next question:

How can we assess learning and provide feedback?

After much back-and-forth, we stumbled upon the following idea. Periodically, each student reflects on his/her own contributions to the group and on his/her own learning. (Sometimes this might be daily and at other times it might be weekly.) We immediately began to craft a rubric, but then another idea hit us. Why not have the students share their own experiences and have the class create their own group participation rubric.

This, of course, led us to our next problem. Ms. Snider teaches some of our best students—students who are highly grade-motivated. Some would undoubtedly grade themselves harshly, while others would unfairly inflate their grades. To counter this, Ms. Snider came up with an outstanding idea: After each self-reflection, lets allow the students time to share their own rating and give the other members of the group time to provide feedback.

For the student who grades herself harshly, this would be easy. For the student who inflates, his/her grade the task would be more difficult for the group mates, but the ability to provide truthful and honest feedback is an important skill. Additionally, this would take some of the pressure off of students having to actually assign their peers a grade (an idea we nixed).  Again, we could use the same student-created rubric to help this process.

While great ideas were flowing from our conversation, we hadn’t yet discovered a way to accurately assess student learning. We were getting closer though.

Last year, I remember watching Ms. Mathews’ students create a Rube Goldberg machine. Along the way, she peppered the students with individual questions, and after presenting their machines, students were asked more detailed questions. Her questions required students to demonstrate their knowledge; their reflective nature also shed light on the entire group experience.

Ms. Snider took Ms. Mathews’ ideas to the next level, “You know what? That’s a great idea.” With increasing enthusiasm, “I think we can go one step further. Why not have the other students in the class ask questions—and I mean real questions after each group’s presentation?”

Finally, we started to discuss two very important individual components. First, all students would be asked to grade themselves according to the class-created rubric. Finally, each student would be required to demonstrate his or her knowledge of the assignment through an additional assessment—perhaps a test or an essay.

We were definitely on to something. As the project progresses, I’m sure Ms. Snider will improve upon the ideas we generated.

What started as a simple conversation morphed into something much more complex. Our focus centered around 4 basic, research-supported premises:

1.    No group grade will be assigned

2.    Students will not be told to grade each other. Students will, however, provide meaningful and honest feedback to their group mates.

3.    Reflection is critical to the learning process.

4.    Targeted, well-crafted and specific questions will be used to assess student learning. This will provide meaningful feedback and can be used as both for both formative and summative assessment.

****

By the way, I need to give Ms. Snider’s project a plug. This is not a group project in which students simply gather facts, arrange them neatly on a poster or PowerPoint, and then spit them out during a presentation. Ms. Snider’s project required students to think for themselves, to pose questions, to think creatively to solve problems and to rely on each other. I can’t wait to see the end results.

How About An Educational Documentary Worth Watching?

On Friday night, like many educators, I found myself watching CBS’s Teach. Within minutes though, I found my mind wandering and my frustration growing. I began to channel surf. Then a friend and co-worker texted me, “Good documentary on CBS called Teach.”

I flipped back, thinking, “Maybe I’m just too cynical about anything created by Davis Guggenheim.”

Admittedly, I wasn’t a huge fan of Guggenheim's Waiting for Superman, so maybe my bias was clouding my judgment. Knowing teachers would be talking about Teach on Monday, I tried to stick it out. I began multi-tasking—if I’m going to watch this I mind as well be productive and learn something. Anything.

For me Teach was too cheeky and almost propagandist. While I have no problem with the Khan Academy, we cannot seriously believe that it’s the solution to our educational issues. The teachers profiled in Teach clearly have made a difference in the lives of their students—as have hundreds of thousands of other teachers.

In an interview with the Huffington Post, Guggenheim paradoxically states, “The strange thing I found is that very few people can tell you what makes great teaching….That’s why we start Teach with the simple premise of ‘What is a teacher, and what do they do.’”

I struggle to agree with the premise that we don’t know what makes great teachers. We do. Research abounds with what makes a teacher great. Additionally, instead of focusing on being great, why not focus on simply allowing teachers to teach well.

Today, too many teachers are handcuffed by overwhelmingly detailed and complex standards with rigid pacing guides. Instead of focusing on real teaching and meaningful learning, time is spent preparing students for the next standardized test.

Teach, in conjunction with Teach.org, kicks off an 18-month campaign to urge students and recent graduates to enter the teaching profession. If that is Teach’s goal, shouldn’t the focus be on schools that have downplayed increasing standardization? On schools that give teachers the freedom to do what’s best? On schools where teachers are respected, purpose-driven and strive to become masters of their crafts?

That would be an inspiring documentary and one worth watching.

So Mr. Guggenheim, how about it? 

Guide by the Side

We can’t expect students to be autonomous and creative when we tell them exactly what to do. Instead let’s aim to point students in the right direction and get out of their way.

I love it when I enter classrooms and I can’t immediately find the teacher. Sometimes the teacher is huddling with students. Other times the teacher is sitting with a group or working one-on-one with a student.

Embodied in this guide by the side philosophy is that students will learn more, discover new concepts and apply their learning on their own. Students are no longer passive learners.  The assignment becomes a quest for knowledge as students strive to discover answers on their own. We’ve moved beyond the transmittal of information from teacher to student. The teacher becomes a facilitator.

Cognitively, this approach makes sense. When students—for that matter anyone—are able to generate relationships between the new material and what they already know, they are far more likely to remember it and apply it. In this constructivist classroom, students are given the opportunity to truly interact with the material.

This is not to suggest that the teacher no longer lectures or instructs. A guide by the side teacher provides a framework, some information, and resources. The guide by the side constantly poses questions to stimulate students.  

Students need to think for themselves, pose and solve problems. In a guide by the side classroom, students produce knowledge instead of reproduce information. As we become guides on the sides—it won’t happen overnight—students will become more independent and real learning will improve.