Another reaction to the recently released PISA results is to suggest that we need more standardized testing in Alberta. Presumably this increase would lead to more accountability.
Some of you who know me well are about to be shocked. I don’t hate our Diploma Exams (12th grade exit exam). In fact, I used to be a big fan of them. These exams are written by Alberta teachers (not publishing companies), are field tested, and then analyzed by psychometricians. They are high-quality exams.
I’m not so fond of the Provincial Achievement Tests given at the end of grades 3, 6 and 9 in Alberta. Fortunately, they are on the way out, and will be replaced by diagnostic exams to be given at the start of those years. The intention is that teachers can use the data to help and support where needed. It’s a great idea in theory. Hopefully it maintains that intent in practice.
The problem with all these exams is that outside organizations started using them for the wrong reasons (ranking schools). Alberta Education made them entirely multiple choice, and the whole thing just wasn’t as useful to me as it once was.
Let’s back up to 1992 when I was a brand-new teacher. The internet wasn’t yet up and running. Email was just starting out. I taught in a small school where I was the only math teacher. There was no Twitter helping me collaborate with other math teachers. There were no blogs to read about how other people were teaching things. I had a textbook, a curriculum guide, and some students. That was it.
The diploma exam was the only feedback I ever got about whether my teaching was meeting Alberta expectations. I loved looking at my results. Some students surprised me. Some students disappointed me. What was really useful, though, was seeing which units went well and which units had room for improvement when I looked at my class data as a whole. I used the results to improve my instruction.
Early on, it did occur to me though, that the diploma exam might be given in the wrong year. The data I got about my 12th graders didn’t help them a bit, because they had moved on. It helped me help next year’s class, but did nothing for this year’s class. Maybe this exam would be more useful if given in 11th grade.
In the late 1990′s, however, outside institutions began using the provincial exam data to rank schools. This exercise is pure statistical lunacy. They didn’t control for poverty, parental education or any other relevant factor. They simply ranked schools. The folly of this ranking can be illustrated by looking at two schools.
Old Scona is in Edmonton, and is the #1 ranked school in 2012 according to the Fraser Institute. I am not criticizing them. It’s a school in my district, and I know some fine people who teach there. They do great things with great kids. Old Scona has selective entrance. They reserve their 120 spots for students whose grade 9 average is over 80%, and they further separate students by having them write a standardized entrance exam. Old Scona selects the best of the best in Edmonton Public Schools. According to the Fraser Institute, the average income of parents of Old Scona students is $103 300 per year. There are no special needs students at Old Scona, and 8.3% of their students are ESL.
Contrast Old Scona with Mistassiny in Wabasca, the Fraser Institute’s last ranked school from 2012 (#279 out of 279). It’s a whole different world. 79.8% of their students are ESL. 20.7% of their students have special needs. The average income of their parents is $30 600 per year. The town of Wabasca is mostly FNMI, and is surrounded by five reserves. This school serves a difficult population. They do not select their students.
If these rankings truly reflect the work being done in these schools by the teachers, then we should be able to swap the entire staffs of these two schools, and within a year or two, Mistassiny would be #1, right? It’s an absolutely ludicrous suggestion.
One year, I switched from a school with a high population of struggling learners to a school with a strong academic population. I taught one class that achieved a 92% average on the Math 30 Pure diploma exam. Only one student in the class failed to get honors on that exam (and he missed by a percent). People were patting me on the back. It was ridiculous praise. I was the same teacher I was the year before in a different setting. The reason, and the sole reason, that my class did so well was because they were really strong students. Any teacher would have had those results with that class. I didn’t suddenly become teacher of the year, simply because I switched schools. I was the same old teacher I always was.
Ranking schools was never the intention of these tests. The data, which could be useful to teachers, is being misused by those with political agendas.
Later on, in 2008, Alberta Education eliminated the written response portion of the Math and Science exams. This was a huge mistake, particularly since it coincided with the implementation of a revised curriculum focused on communication and personal strategies. You can’t assess those things on a machine scored test. The written questions provided students with the opportunity to demonstrate their strategies. The written questions were a great way to lend value to important parts of our curriculum like communication and problem solving. Losing them made the exams worse.
Further, I have come to understand that Alberta Education is now exploring ways to have English and Social Studies writing scored by machine as well. As I understand it, these guys are working with Alberta Education to come up with a way to mark the writing of Alberta Students.
Over the years, I have taught more than 30 classes that wrote diploma exams. I loved teaching those classes. I didn’t teach to a test. I taught the curriculum to the best of my ability, and the test results always seemed to come out all right. There were some classes and some students that did better than I expected. There were some classes and some students who did worse than I expected. I always looked at the data, though, and tried to figure out what I could have done better.
I don’t think provincial exams are a terrible evil. I do think the data are being misused. I do think we should consider moving the grade 12 exams back a year so we could still help those students who need it. I definitely think we need to re-institute written response on math and science exams, and leave the scoring of all writing to actual human beings.
Read Full Post »