Feeds:
Posts

## Homework Research

In my last post, I speculated that there were three reasons I read educational research.

1. I encounter it (via Twitter, blogs, or in journals) and I’m curious, so I read it.
2. I deliberately seek it out to confirm a bias. (Don’t judge me. We all do this.)
3. I’m genuinely interested in what the research has to say on a certain topic, so I search for it.

Since biases are fun, let’s look at an article I dug up for the second reason. I’ve made my views on homework pretty clear on this blog in a couple of posts. Here’s a study I found on the subject of homework. Unfortunately, it failed to confirm my bias.

Are We Wasting Our Children’s Time By Giving Them More Homework?

The study is by Daniel J. Henderson, of New York and was published as IZA Discussion Paper No. 5547, March 2011.

Following an identification strategy that allows us to largely eliminate unobserved student and teacher traits, we examine the effect of homework on math, science, English and history test scores for eighth grade students in the United States. Noting that failure to control for these effects yields selection biases on the estimated effect of homework, we find that math homework has a large and statistically meaningful effect on math test scores throughout our sample. However, additional homework in science, English and history are shown to have little to no impact on their respective test scores.

Yikes. Math homework has a large and statistically meaningful effect on math test scores throughout our sample? Uh. Oh. I guess I’d better read more than just the abstract and see if I can figure our what is going on. The math used in the study is complicated. That might make it tricky to read.

Here’s something I wonder about. Page 9.

…higher able students benefit more from additional homework.

Perhaps higher able students are the only ones who actually do the homework, because they’re the only ones who are capable of doing it.

Later, on page 17.

Taking the Peabody Individual Achievement Test in math as our benchmark, the gain from math homework (1.77 points) corresponds to one-fourth of the raw black-white test score gap between the ages of 6 and 13

My question would be: Can we be sure the gain on that test is solely attributable to homework? Maybe we can. I’ll admit to not fully understanding the tables in the study.

Here’s a finding on page 19 that I am glad to hear. At least one of my biases was confirmed by this study.

The teachers Treatment of the homework (whether it is being recorded and/or graded) does not appear to affect the returns to math homework.

## Educational Research

I’m going to write some posts over the next little while about educational research. Just before Christmas, Michael Pershan and Chris Robinson were going back and forth on Twitter about research vs. blog posts.

“If teachers can rely on blog posts, where does that leave ed research?”

That question got me thinking a whole lot about how and why I use educational research and how and why I use blog posts.

I read research for several reasons.

1. I encounter it (via Twitter, blogs, or in journals) and I’m curious, so I read it.
2. I deliberately seek it out to confirm a bias. (Don’t judge me. We all do this.)
3. I’m genuinely interested in what the research has to say on a certain topic, so I search for it.

My recent blog post about delayed feedback falls into the first category. A colleague showed it to me and I was curious, so I read it.

I tend to mine blogs for ideas that I can use immediately in classrooms and workshops. Those ideas don’t have to be researched based, in my opinion. The fact that a colleague tried it already and it worked for her is sufficient for me to try it out. That endorsement is worth one class period or one unit of study of my time. I see these shared ideas the same way I saw lunchroom conversations in the 1990s. “I did this cool thing in my class today. You should try it out.

If I were contemplating a major shift in my practice, I’d probably go to research in addition to listening to colleagues. SBG would be an example of something I’d research before changing my whole practice. A blog might inspire me to try it, and the research would confirm that it’s worth doing. One year in Math 8, I did the entire course in cooperative learning groups and activities. That’s a big commitment. That’s a big shift. Research supported and justified my change.

In the next few blog posts, I’m going to look at some of the research I’ve read over the past few years. I’ll explain how I happened across it, and how I use it now.

## Delayed Feedback

It’s been an interesting enough week in the assessment world that I’m compelled to blog for the first time in a long time.

Early last week, I encountered this “Focus on Formative Feedback” literature review by Valerie Shute.

Table 4, near the end, on page 179 lists “formative feedback guidelines in relation to timing issues.” Shute recommends using immediate feedback for difficult tasks, and delayed feedback for simple tasks. She says that to promote transfer of learning, teachers should consider using delayed feedback. To support that claim, she says,

According to some researchers (e.g., Kulhavy et al., 1985; Schroth, 1992), delayed may be better than immediate feedback for transfer task performance, although initial learning time may be depressed. This needs more research.

Then, just yesterday, Dan Meyer jumps in with a post on delayed feedback.

My gut says that the timing of the feedback is far less important than the quality of the feedback. Dylan Wiliam has entire chapters dedicated to providing feedback that moves learners forward. Next steps are useful to all students. Evaluative feedback that evokes emotion isn’t particularly useful to anyone.

I’m not sure this does need more research.

## The Sky Is Falling!

There’s been a lot of twitter and media buzz about a new app that scans math questions and gives answers.

Dan Meyer has compiled some thoughts on the app over on his blog, and he has been commenting on Twitter as well.

I decided to test Dan’s comment with (what else?) a test. I gave the app one exam from each of grades 7, 8, 9, 10, 11 and 12. My conclusion is that it doesn’t solve them anywhere near as well as most kids would.

Full disclosure: The grades 10, 11 and 12 exams were ones I created, and I’m always conscious of trying to avoid having questions that can be answered with a calculator alone. The grades 7, 8 and 9 exams were from a publisher.

I tried to pick topics it had a shot at solving. I tried to pick topics with mostly number and equations.

Grade 10 – Algebra and Number

The app got 0/30 on my exam. On the questions I thought it should be able to answer, it got 0/10.

This one was its most blatant error. I did have it centered properly prior to snapping a screenshot. It registered the 2, and ignored it.

These were its first steps. It had trouble recognizing that square bracket.

The app got 3/32 on my exam. On the questions I thought it should be able to answer, it got 3/13. I was a little surprised. Clearly I need to tweak some questions. Here’s one it got right.

Grade 12 – Exponents and Logarithms

The app can’t recognize logs, or manipulate anything but the most rudimentary equations. It got 0/30 overall and 0/9 on the ones I thought it should get.

Math 7 – Integers

The app struggles with brackets. I hovered over expressions like (-2) + (4) – (-7) endlessly waiting for an answer of any kind (right or wrong) and never got anything. It got 0/20 overall and 0/9 on the ones I thought it would get.

Math 8 – Fractions

The app nails fraction calculations. It got 7/20 overall and got 7/7 on the ones I would have expected it to get. Here’s one it got right.

Math 9 – Equations

The app got 6/20 overall and 6/6 on the ones I would have expected it to get. It solves basic equations (no logs, no powers, no quadratics, few brackets) correctly every time I try it. Some of the steps seem convoluted to me.

I’m not sure this is the game changer some people fear it is. It’s a calculator, and not a particularly accurate one. As long as we’re asking the right questions, let them use this app. Just have them check their answers on a calculator.

## My AAC Work

My last post reflecting on my two and a half years at Alberta Assessment Consortium was too much about feelings and not enough about number crunching. Here are some numbers reflective of what I did.

Travel

• Total distance driven = 39 126 km
• Total distance flown = 13 962 km
• Nights in hotels = 97 (48 this school year alone)

Far too many of the drives this winter looked like this:

With all that driving, my 160 GB iPod and its 7824 songs was my best friend.

• Most of the time I play it on shuffle mode, all songs in the queue.
• The top 25 most played is a diverse list including Adele, Biz Markie, Edwin Sharpe, Pitbull, Gwen Stefani, Leonard Cohen, Project Jan & Project Jenny, Shaggy, Mumford & Sons, Band of Horses.
• The most played song (81 plays) was Whale of a Tale by Danny Michel.
• On shuffle, a lot of songs end up coming up that I’m not in the mood for.
• The most skipped song (50 skips) was something called April Showers by Sugarland. I wonder why it’s on my iPod.
• High on both lists are Hate Me by Blue October (39 plays, 37 skips) and A- Punk by Vampire Weekend (37 plays and 37 skips)
• I worked my way through all the Freakonomics podcasts from start to finish.

Cities and Towns Visited For Work

• 29 Different Cities
• Most Visited City – Grande Prairie – 30 Days
• Second Most Visited City – Fort McMurray – 19 Days
• Closest City Visited – St. Albert (or is Sherwood Park closer?)
• Farthest City Visited – Toronto, Ontario

Work

School Visits

• 153 School Visits
• 42 Unique Schools
• Grande Prairie Composite was stuck with me the most, at 23 visits.

Coaching Visits

• 85 Coaching Visits
• 41 Different Teachers Coached

Workshops/Presentations

• Total, including full day, half day, and shorter – 93
• Teachers in workshops – 2017
• Unique teachers in workshops – A subset of that 2017
• Workshops in French – 5
• Most common workshop theme – Formative Assessment, of course.

Meetings

• Meetings Attended – 121 (Ug!)

Demo Lessons

• Total – 43
• Total Flops – 2

Meals With Keynote Speakers

• Steve Leinwand – 1 (But it was actually the second time I dined with him)
• Cathy Lassiter – 1
• Ruth Sutton – 2
• Ken O’Conner – 1
• David Coffey – 1
• Kathryn Coffey – 1

## What I Learned At AAC

My secondment at Alberta Assessment Consortium ends next week. For the past 2.5 years, I have traveled the province conducting a research study in which I worked with math teachers on embedded formative assessment. We also studied the coaching model as a professional learning tool.

As I transition back to my district, I’m reflecting on my time at AAC. I’d like to share with you what I think I took most from this experience.

I could tell you about all the people I met across the province who are doing great things in high school math classrooms, but that would sound trite.

I could tell you about how much I learned about assessment, but I’d have been doing an absolutely terrible job of this work if I didn’t learn a whole lot.

When I took the job, I had no idea I would need to make videos as part of the project. The ones I made are posted here. They’re not in the order I made them, but an astute viewer will see my progression. After the first one, we bought new camera equipment because the flip camera wasn’t cutting it. At one point, we had a videographer come in and teach us about cuts, B-roll, transitions, multiple cameras and other tricks. We hired a video “intern”, who made one video for me, and helped me dabble in Adobe. For the most part, though, those videos are all me, and are all iMovie.

The thing is, I had no idea I’d enjoy that creative process so much. Let me tell you how much I enjoyed it.

Last week, I spent a day at a local elementary school filming K-3 students talking about their writing. I hit it with three cameras, one on a boom giving an overhead shot of the students’ work. I recorded an audio track on a separate microphone. I brought a colleague to interview the students so I could focus on filming. I did my best to film it like a pro. In the end, I had more than 90 minutes of footage, filmed from three different angles. This footage is to be used by our video intern under the guidance of future AAC employees to make 30 second snippets to use in workshops and to post on our website.

The thing is, I couldn’t let it go.

Even though I don’t own the footage, and can’t use it myself, I had to make something from it. Knowing full well that no one would ever see it outside our office, I spent hours piecing it all together into something I loved. It’s 15 minutes of young kids talking about feedback. I built in multiple angles. I worked in their funny comments. I worked in their insightful comments. I pieced it all together in a manner that really amuses me. I added transitions and pulled audio tracks from my best track into the clips from the other cameras. I learned how to line that audio up to the students’ lips. It comes in at 15 minutes long, and it’s some of my best work. I’ve revised it twice more after rendering it and showing it to people.

On Friday, I’ll wipe my work laptop clean and pass all my video (including this one) on to the boss on a hard drive. At that point, I won’t even have a copy of this creation any more.

Why did I do all that knowing that very few people would ever see it, and that I couldn’t keep it? Because it reflects the thing I learned most about and really enjoyed doing during this job. Who (other than that Bloom guy) knew that a creative process could be so enjoyable and valuable? That’s a nice thing for a rigid math guy to come to understand.

## Old Friends and 101qs

Around about 4th grade in the early 1980’s, I was walking home from school and saw the new kid in my class following the same path. We got talking and realized that he lived just around the corner from me. The proximity law that governs childhood friendships dictated that Andrew and I would become best friends.

Andrew and I had a lot in common. We were both smart (although too modest to tell people that, until today). We both loved reading trashy horror novels (Stephen King and Dean Koontz were our favourites). As a team, we rocked the junior high debating circuit in Alberta for a couple of years.

Andrew and I grew apart a bit in high school, and then he moved to Ontario. I saw him once in the early 1990’s when I drove across Canada. A year or two later, he showed up at my parents’ house one day when I just happened to be there. Five or six years ago, one of us managed to track down the other, despite Andrew’s aversion to Facebook. We correspond infrequently by email, and occasionally update each other about our families. Last weekend, I got an email from Andrew. He is not a math educator (I think he does government training), but he reads my blog. He sent me this:

Source: http://imgur.com/JDZ1WDk

That’s the thing about curiosity and perplexity in mathematics. You don’t need to be a math teacher to know a good 101qs when you see one. Everybody wonders about things that can be explored mathematically.

I wonder if Justin lost or gained followers when he got arrested.

I wonder if this growth is linear. I wonder if I could have a couple more data points.

I wonder when and if Kim will pass Justin.

I wonder why Andrew knows more about what I do than I know about what he does. I’m a terrible friend.

## SUM2014

Last week, I had the privilege of being allowed to take my road show to Saskatoon, SK. I presented at, and more importantly, I got to attend SUM2014, the annual conference for math educators in Saskatchewan.

It came at a time when I really needed it. It was a great two days hanging out with math educators. I learned a lot, and had a lot affirmed.

Steve Leinwand‘s keynote was a joy. If you haven’t had the pleasure of hearing him speak, check out this presentation (from another conference). As Dan Meyer says, “this guy breathes fire.”

Other highlights for me were:

• Reconnecting with David Coffey and Kathryn Coffey, who I first met two years ago in Edmonton.
• David and Kathryn’s session on literacy and math. We’re into that here in Alberta, too, so it was timely.
• Meeting Nat Banting in person. Watch this kid. He’s a rising star in math education.
• Reconnecting with Park Star. I now know (and remember) her real name, but it’s more fun to pretend I don’t.
• Meeting Lisa Lunney Borden. We only had a few moments to chat over breakfast and before the conference started, but now I know about CMESG, which I think I will attend.
• Briefly disengaging from David Coffey’s session on engagement and convincing the woman beside me to join Twitter.
• I enjoyed my sessions. Some of the participants were kind enough to let me know they did too. That kind of feedback is always appreciated. Keep in touch.
• Having supper with Steve, Anne, David, Kathryn, Nat, Michelle, Jacquie and Allison. It was a great meal, and I got Leinwand’s ear to myself for a bit. His bloggable advice on the current math debate is to build bridges. Connect with math professors. Listen to each other. He’s wise. He’s been through this before. I appreciated him listening to me.
• Steve Leinwand referred to me in the closing session as “That dude from Alberta.” I felt like I had arrived.
• A panel discussion with (L-R) Kathryn, David, Me, Steve. Terry Johanson was also on the panel, but we took this before we started (notice Steve’s engagement level), and she wasn’t there yet.

We ended with Steve Leinwand modelling practice. One of the panel questions was “How do you coach or teach subversively?” We all answered (except me – my voice was gone). Then Michelle was wrapping up. She asked the audience if they had any questions. They didn’t. There was time left. Steve jumped up, asked the audience to take 2 minutes to share their conference “take-aways” with a neighbour. Then he asked them to share back with the whole group. One person shared back something he learned in my session, thereby earning a beer on me next time I’m in town. Other people shared what they learned. Steve took over the wrap up and modelled a large group reflection. That’s subversive coaching, right there, folks.

## 2013 in review

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 36,000 times in 2013. If it were a concert at Sydney Opera House, it would take about 13 sold-out performances for that many people to see it.

## Book List

I haven’t read this book, but I intend to, based solely on the quote below. I’m intrigued.

There are some kids growing up with way too much adversity in their lives and what they need more than anything is protection from that adversity. And then we have other kids, especially kids who grow up in affluence, who just don’t have enough adversity in their lives. I think that is a hard message for parents to hear… In trying to protect them from adversity, we can sometimes be doing more harm than good.