Post Category: Math

Musings on mathematics, logic, and abstract thinking.

A Math Major on Khan Acadamy's Exercises

  • By Brad Conte, December 13, 2012
  • Post Categories: Math

I finished all of the practice exercises on Khan Academy. I have to say, it was fun.

Profile bar after finishing all exercises (My Khan Academy profile)

I'm not exactly not their target audience, though. I received a B.S. in pure math about 3 years ago, so my math background is far beyond the current content offering of Khan Academy (which is basically U.S. high school AP).

I was motivated to try the exercises for a couple reasons. First, I hadn't really checked out Khan Academy before, despite the fact that it had generated a lot of interest on the Internet over the last couple years, I saw it referenced a lot as a math review resource, and I heard undergraduate engineering students swear by it. Second, I'm a big fan of doing consistent mental exercises. I definitely exercise my brain during the work/reading/fun cycle, but I like having some sort of consistent activity to warm up my brain. I usually prefer puzzles that require focus but don't stretch your brain. I hadn't had a set of such exercises in a while and I thought that basic math review would work well for that purpose.

I did all the exercises and watched a small handful of videos. My comments are only about the exercises, not the videos.

Doing the Exercises

I worked through the exercises by following their exercise dependency tree. It's an inter-connecting tree of all the exercises and how they relate to each other. The top/root of the tree starts with an exercise on basic one-digit addition, branches out through the rest of arithmetic, continues into geometry, algebra, etc, and then eventually collapses down to Differential Calculus. There are 37 groups, each with between about 3 and 20 individual exercises. From the zoomed out perspective you can see how all of the high-level groups relate to each other and if you zoom in you can see how the exercises within each group inter-relate, with a few of the edge exercises connecting to exercises from nearby groups.

I followed the tree very methodically from the top down, working left-to-right when a tie-breaker was needed. Exercises are presented in sets of 8 questions and you earn "proficiency" at an exercise by demonstrating sufficient skill at the exercise. You can choose to do problems just from one specific exercise or a mix-up of questions from all the exercises in the group. I chose to do just one exercise set at a time because I enjoy getting into a focused "zone" while solving problems and I was largely doing this as a brain warm-up. I'm not sure it was best, though, if I continue doing exercises I will probably use group questions.

I worked through the exercises at varying rates. In the beginning I would usually do exercises for a couple 10 minute sessions over the day, one in the morning and another one at lunch or during the mid-afternoon. After a couple weeks I enjoyed the process more, especially as I moved past the elementary exercises and into the more fun ones like probability and systems of linear equations. I had never used a math question-answer format like this before, being able to get a stream questions in an easy-to-answer format was downright fun. I found myself putting more free time into the exercises. After all, who doesn't love to do some math problems here and there? (It was a rhetorical question. The majority of you can put your hands down now.)

As expected, the exercises weren't challenging. As a math major, I was not only well-trained in mathematical thinking but I also used the principles of geometry and algebra constantly, so there was nothing there I was not familiar with. However, a handful of exercises used tricks (like converting repeating decimals into fractions) or identities (like trig identities) that didn't come to mind and prompted me to use the "I'd like a hint" button to see a solution strategy.

To make the exercises more engaging I restricted myself to not using scratch-work or a calculator whenever it was possible. This made some of the exercises more difficult and lead to various mistakes in some of exercises requiring multiple steps. It was a good challenge, though, and some of the exercises forced me to keep more objects in working memory than was comfortable. I actually had a few exercises where I constantly made mistakes due to bad mental work, typos, or hasty shortcuts (such not bothering to check whether 221/299 can be reduced by a factor of 13).

While I worked from start to finish fairly linearly, I didn't race through the material. I backed up and re-did exercises that I found particularly fun and re-did some exercises using slightly different methods. (Although I'd estimate only 1/5 of my total points came from re-doing exercises, I think they are worth fewer points once you are proficient at them.) I finished all 380 exercises in about 2 months of consistent work. Khan Academy is constantly adding content, so I plan to do future exercises they add.

The Educational Experience

Although I couldn't see the exercises from the point of view of a student with little mastery of the content, I had some thoughts on their educational value.

I think that the biggest benefit of the Khan Academy exercises is that they can supply an unlimited number of practice problems and provide immediate feedback and explanations. This is something that's hard to do in a non-computer setting. A teacher can only spend time on so many examples during class and homework doesn't give a student immediate feedback. The strength of automation is that examples are infinite and feedback is instant and I think they leveraged both of those aspects well.

The tree of interconnected math concepts was well done. I wish I knew of a similar detailed tree for higher math topics. A very select few of the tree relationships did seem backwards, where I had a hard time believing that the a latter exercise offered any challenge a former exercise didn't. (I now wish I had saved examples, but I didn't.)

The exercises did a good job of focusing on one thing at a time. When learning any new skill it usually helps to isolate it in a familiar environment and focus on the unfamiliar part, so by focusing only on one idea per exercise these exercises would likely be a very helpful learning aid for someone who is trying to improve a specific weak point in their math skills. Combined with the tree as a whole, it would probably be easy to backtrack from an exercise full of confusing ideas to the lowest point of weakness and then work on refining the relevant skills from the bottom up. I can see a coach being able to do this easily for a student, or a motivated student doing it them self.

The system uses some clever math and machine learning to estimate when students are have achieved proficiency in an exercise (but all the student sees is a progress bar). Not knowing this when I began, it was still immediately clear that solving the first several problems in a set quickly and accurately earned proficiency but a couple of mistakes required numerous correct problems to earn proficiency. The system was well-tailored and let an initial burst of obvious competence pass quickly, while not passing mostly-right-but-still-struggling performance.

The "I'd like a hint" button was not the learning resource I hoped it would be. It offered step-by-step solutions (allowing the user to reveal one step of the solution at a time) but the steps were very formulaic without much explanation. For example, a typical step would read like "next we take the X and blah it with the Y", but with no mention of why this was necessary or possible. The exercises definitely were not teaching resources by themselves. Each problem did have a link to the associated teaching video, though, so they weren't designed to stand-alone.

Here are some of exercises that I thought were particularly interesting:

  • The group of triangle proofs. This set allows for a unique chance to walk through geometry proofs step by step. The way they set it up, each step was verified as you input it, so it was pretty much impossible to get it wrong. The geometry diagrams would light up the relevant portions each time you input a new step or hovered over an old step, so I think it would give nice feedback for students who have a harder time "seeing" the geometrical aspects of what they're doing.

  • The exercise on derivative intuition. A simple introductory concept that was well-illustrated. I'd recommend all beginning calculus students do this exercise. There were a couple other exercises oriented at building "intuition" for a topic. I really liked this, since I'm in favor of teaching intuitive concepts. (These should be replaced by rigor later on, of course, but many students, such as myself, benefit from these sorts of "gimme a clue or view as to what's happening here" explanations.)

  • The group of logical reasoning. A quick review of the basic steps of logic and the relevant syntax. This is typically covered in the first few weeks of a first-year introduction to math college course for math or computer science students. I'd recommend everyone in such a class go through the exercises to help cement those ideas.

Unfortunately, some of the exercises lacked diversity in the problem sets. As examples, when reducing fractions there were never common factors larger than 13 (possibly to keep it mentally doable, though), polynomials often seemed to be picked out of only a few sets of styles, and there were a limited number of triangle ratios used. Some of the problem sets were constrained by the intent of the problem, such as trying to pick a problem that resulted in clean answers, and those constraints left only a handful of possible starting values. Those were understandable, but some of them didn't have any such constraints. With problems that didn't vary much, a student could unconsciously get into a rote system of "take the number there, blah it against that number, blah the result and write it down". The entire point of Khan Academy is self-study so I'm not suggesting it needs to take anti-cheating measures, but there were some problems sets that seemed unnecessarily bland considering the pool of potential problems. With some of those exercises I would actually consistently get the exact same question twice within a span of eight questions.

There weren't many exercises on the deeper/more challenging topics. The practice exercises don't cover as much depth as the video lessons do and I would like to see more of the topics fleshed out. For example, there are very few Linear Algebra exercises yet many Linear Algebra videos. Calculus is limited to Differential Calculus only. Even in Algebra they covered a lot of specific skills, but left out some worthwhile ones. They're consistently adding new exercises, though, (they've added 13 exercises in the past three months alone) so hopefully these areas will be expended in the future.

Miscellaneous Observations

On grading:

  • You can achieve proficiency at any point during the exercise set, not just the end. You can keep an eye on the green star in the left-hand bar to see when you get it. Once achieved it can't be lost, although it can be recommended for review.
  • I could almost always pass a problem set in four questions if I answered them correctly and within their "fast" time slot.
  • The most efficient strategy that I found for passing a problem set was to get the first five questions right with three of them done at "fast" speed. Focusing on speed for all four would encourage hasty mistakes. Better to let the easy ones be fast and give them all good attention.
  • Once a mistake was made, it would take at least 6, usually more, correct answers to pass the set. Wrong answers were weighted much more negatively than slow answers.
  • A mistake seems to set your progress bar for the exercise down to at most half-filled.

On answers:

  • Their scratch pad interface is a nice idea, but it's impractical to draw/write things with the mouse if you need to be fast. I just used a pen and sticky notes when I wanted scratch paper.
  • Allowable answer formats varied somewhat inconsistently. Fractions almost always had to be in reduced form, but not always, rounding varied from 0 to two decimal places, and these were across exercises that had nothing to do with reducing fractions or rounding decimals. It was worth checking their little "acceptable format" indicator to see what types of input was allowed.
  • The answer parsing was reasonably flexible. For example, when entering multiples of pi you could write "5pi" or "5 pi". It was flexible enough to not be annoying.

Final Thoughts

The Khan Academy exercises are a good source of practice material. They are not, however, a learning resource and do not replace real homework assignments and feedback by teachers. But I think they're solid supplementary material. I also think they would help someone who was once comfortable with material and wants to re-learn it.

I think Khan Academy did a good job making the exercise format pain-free and enjoyable.

Thoughts on "Changing School Mathematics"

  • By Brad Conte, September 21, 2012
  • Post Categories: Math, Reading

I read an interesting essay about the "New Math" effort from several decades ago: Changing School Mathematics by Robert Davis (about 12 pages long). The essay gave an overview of New Math but focused on the Madison Project, a specific effort at implementing New Math, and how it sought to change the way students learned math from the old style of following formulas to a new style of emphasizing conceptual understanding. The essay covered some interesting techniques of teaching and how students responded, and I some of the thoughts from the essay really resonated with me.

The project was set up to address some shortcomings in the popular math teaching of the time and the difficulties that math education faced. One of the biggest problems in teaching mathematics, particularly at the primary education levels, is that it is both difficult to convey good understanding of mathematics and to learn if the student has gotten a good understanding or if they are simply memorizing and repeating steps. Much of the usefulness of math stems from understanding what is happening and why it is happening and a student who doesn't understand that is missing out on the majority of what learning math has to offer. So there has been a lot of debate and experimentation about the "right" way to teach math and what styles are or are not effective.

I liked a lot of what the author had to say about teaching math, and I think a lot of it applies to learning and teaching in general. A lot of it resonated with my inner mathematician (I'm a math major) and reminded me of ways that I like to learn and ways that I think are effective in tutoring other people in math. (Teaching math is a subject I'm very interested in right now, as I think about starting to teach my children math in the next few years.)

Here are some interesting quotes from the article, although I recommend reading the whole thing. I think that a lot of it applies to most learning topics in most learning scenarios, not just math in a class room. (Emphasis in quotes is preserved.)

No child can be expected to "discover" historical accidents or what is in the teacher's mind. Only after a task is clearly understood can the creativity and inventiveness of children take over the agenda. The correct understanding of the [Madison] project's approach might better have been stated as, "If, at this moment in the lesson, what is needed is an intellectual breakthrough of some sort, please wait and let a student take the first step." If you wait, someone will.

- pg. 637

I like this point because I'm very convinced that we both understand and remember things better when we make the connections ourselves. It's certainly possible to feed people answers prematurely, either for the sake of keeping up a pace or because we're too excited ourselves and we want to share our knowledge, and in doing so we hinder the listener's ability to play with key pieces of the puzzle themselves. Providing an answer too soon can deny the listener a "light bulb" moment, which is key for understanding ideas.

When I'm explaining an idea to somebody, perhaps one that itself involves several different ideas, there will inevitably be a time where a few ideas come together and the listener doesn't instantly make sense of everything. (This arises frequently, almost whenever a relatively deep conversation has gone on for 10 or so minutes.) They're not dumb, just taking a moment to sift through things. When I feel like this has happened, I usually like to just go quiet for a moment and let the other person think. Even if they ask a question, I may just ignore it for a little bit (politely, of course, perhaps with a little "hm..." and a pregnant pause). The idea will make more sense to them and be more useful if they put it together in their own head, and I'm usually certain that nothing I can say will substitute for their own effort.

Besides demonstrating the assimilation paradigm, the preceding example shows another way in which the project provided help to students: the use of clear, unambiguous language and notations. We had been aware that David Page was using small raised symbols for positive and negative, as in +2 or -3, and carefully using the words positive and negative when that was the idea (as opposed to plus and minus in the situation where those meanings were intended and nonraised symbols were used). We had not chosen to follow his example in this usage until some seventh graders, asked to invent a sensible way to add, subtract, and multiply signed numbers, responded that

(+2) x (-3)

should be equal to 3. How come? "Because two times three is six, and then you have to subtract three." We converted immediately to Page's notation, and this difficulty disappeared.

Probably, if we were telling the students what to do, alternative interpretations such as the one above might not arise --or, at least, might not see the light of day and might not come to be noticed. But if the children themselves are building up the mathematics, if they are inventing ways to proceed, then exactly how they are thinking about the ideas becomes directly relevant. In traditional "teaching by telling," the question of which notation is used may be seen as unimportant.

- pg. 640

When a teacher is emphasizing a conceptual understanding of math, clear notation is necessary. This example looks like a very interesting case where seemingly simple notation obscured the math problem. The student tried to derive a mathematical meaning from the notation, but the notation was imperfect. It was a simple example, and the author points out that this is easily solved by just telling the student what to do (they will eventually memorize the rules through repeated practice), but it illustrates how notation can be an artificially imposed difficulty.

I think that good notation should distinguish between what something is and what actions are happening. There is probably a more formal way of expressing that thought, but that's my basic idea. In the example above, the student confused what quantity existed (negative 3) for what action needed to be done (subtracting 3). Obviously, the two ideas are closely related, but technically the notation (aka, "-3") was ambiguous by itself, the intent had to be inferred from the context, and the interpretation mattered. (This last point is important because, particularly in higher math, ambiguous notation may be permitted when various interpretations/definitions are equivalent, so the reader is free to choose the interpretation they like best.) Such seemingly simple things can, at a minimum, create confusion, and once the student is confused their progress will be slowed down. I don't really see a need for such ambiguous notation, and in an ideal system we wouldn't have it. Since notation is mostly established by popular tradition, it's no surprise that odd quirks like this exist.

The essay closed with:

By imagining that mathematics means knowing when to invert and multiply, we have come to trivialize mathematics, knowledge itself, and even the nature of human thought. Anyone who listens carefully to what children really think about the world will know otherwise.

-pg 645

Another re-iteratation of the thesis: Students, even young ones, can understand ideas and find motivations for those ideas on their own given sufficient guidance. Rote memorization of using tools doesn't teach the same skills.

Math can be a difficult subject to teach. One reason it's hard is because a good understanding of it is generally a very "internal" feeling. Mere words don't really convey what a person understands about math, much like words fail when trying to describe a piece of music. Differences of viewpoints, preferred approaches, and mental strengths between teachers and students can raise barriers when trying to take knowledge from the teacher's head and get it into the student's head. But it's not an impossible task, and there are definitely some techniques have tend to produce better success rates than others. Good teaching will allow the student to build math understanding in their own head, instead of cramming a pre-defined structure in there. The Madison project tried to accomplish this, and it seems like they had a lot of good ideas that students of any generation would benefit from.