Feeds:
Posts
Comments

## The Sky Is Falling!

There’s been a lot of twitter and media buzz about a new app that scans math questions and gives answers.

Dan Meyer has compiled some thoughts on the app over on his blog, and he has been commenting on Twitter as well.

I decided to test Dan’s comment with (what else?) a test. I gave the app one exam from each of grades 7, 8, 9, 10, 11 and 12. My conclusion is that it doesn’t solve them anywhere near as well as most kids would.

Full disclosure: The grades 10, 11 and 12 exams were ones I created, and I’m always conscious of trying to avoid having questions that can be answered with a calculator alone. The grades 7, 8 and 9 exams were from a publisher.

I tried to pick topics it had a shot at solving. I tried to pick topics with mostly number and equations.

Grade 10 – Algebra and Number

The app got 0/30 on my exam. On the questions I thought it should be able to answer, it got 0/10.

This one was its most blatant error. I did have it centered properly prior to snapping a screenshot. It registered the 2, and ignored it.

These were its first steps. It had trouble recognizing that square bracket.

Grade 11 – Radicals and Absolute Value

The app got 3/32 on my exam. On the questions I thought it should be able to answer, it got 3/13. I was a little surprised. Clearly I need to tweak some questions. Here’s one it got right.

Grade 12 – Exponents and Logarithms

The app can’t recognize logs, or manipulate anything but the most rudimentary equations. It got 0/30 overall and 0/9 on the ones I thought it should get.

Math 7 – Integers

The app struggles with brackets. I hovered over expressions like (-2) + (4) – (-7) endlessly waiting for an answer of any kind (right or wrong) and never got anything. It got 0/20 overall and 0/9 on the ones I thought it would get.

Math 8 – Fractions

The app nails fraction calculations. It got 7/20 overall and got 7/7 on the ones I would have expected it to get. Here’s one it got right.

Math 9 – Equations

The app got 6/20 overall and 6/6 on the ones I would have expected it to get. It solves basic equations (no logs, no powers, no quadratics, few brackets) correctly every time I try it. Some of the steps seem convoluted to me.

I’m not sure this is the game changer some people fear it is. It’s a calculator, and not a particularly accurate one. As long as we’re asking the right questions, let them use this app. Just have them check their answers on a calculator.

Advertisements

### 2 Responses

1. Hi John,

I love it when my students “cheat” on assignments. Almost invariably these attempts drive them to invest more effort rather than less. Along the way they wind up teaching themselves far more effectively that I can teach them.

I have a particularly onorous assignment I give my classes, challenging them to be the first one to get all of the answers correct. I tell them to use whatever resources they have available to them and then I retreat to my office.

In most of my classes there will be one or two students who figure out that I am one of the resources they have to work with and they track me down. I answer all the questions they put to me – but then I tell them that I have purposely given them some incorrect information. Word invariably leaks that I’m willing to help, so all of them ultimately come to find me.

What follows is a frenzy of fact-checking as they try to find which information I’ve given them is inaccurate. They emerge as subject matter experts. Meanwhile I’ve enjoyed the opportunity to catch up on my backlogged emails.

The best part is that I don’t actually give them bad information at all. Yes, it’s a teeny bit evil, but a lot of fun.

I don’t teach math, and my students are all self-motivated adults – so I’m not entirely sure this is relevant to a proper classroom – but there you go.

2. […] 2014 Oct 23. John Scammell puts PhotoMath to work on tests throughout grade 7-12. More disaster. […]