I finished all of the practice exercises on Khan Academy. I have to say, it was fun.

I'm not exactly not their target audience, though. I received a B.S. in pure math about 3 years ago, so my math background is far beyond the current content offering of Khan Academy (which is basically U.S. high school AP).

I was motivated to try the exercises for a couple reasons. First, I hadn't really checked out Khan Academy before, despite the fact that it had generated a lot of interest on the Internet over the last couple years, I saw it referenced a lot as a math review resource, and I heard undergraduate engineering students swear by it. Second, I'm a big fan of doing consistent mental exercises. I definitely exercise my brain during the work/reading/fun cycle, but I like having some sort of consistent activity to warm up my brain. I usually prefer puzzles that require focus but don't stretch your brain. I hadn't had a set of such exercises in a while and I thought that basic math review would work well for that purpose.

I did all the exercises and watched a small handful of videos. My comments are only about the exercises, not the videos.

## Doing the Exercises

I worked through the exercises by following their exercise dependency tree. It's an inter-connecting tree of all the exercises and how they relate to each other. The top/root of the tree starts with an exercise on basic one-digit addition, branches out through the rest of arithmetic, continues into geometry, algebra, etc, and then eventually collapses down to Differential Calculus. There are 37 groups, each with between about 3 and 20 individual exercises. From the zoomed out perspective you can see how all of the high-level groups relate to each other and if you zoom in you can see how the exercises within each group inter-relate, with a few of the edge exercises connecting to exercises from nearby groups.

I followed the tree very methodically from the top down, working left-to-right when a tie-breaker was needed. Exercises are presented in sets of 8 questions and you earn "proficiency" at an exercise by demonstrating sufficient skill at the exercise. You can choose to do problems just from one specific exercise or a mix-up of questions from all the exercises in the group. I chose to do just one exercise set at a time because I enjoy getting into a focused "zone" while solving problems and I was largely doing this as a brain warm-up. I'm not sure it was best, though, if I continue doing exercises I will probably use group questions.

I worked through the exercises at varying rates. In the beginning I would usually do exercises for a couple 10 minute sessions over the day, one in the morning and another one at lunch or during the mid-afternoon. After a couple weeks I enjoyed the process more, especially as I moved past the elementary exercises and into the more fun ones like probability and systems of linear equations. I had never used a math question-answer format like this before, being able to get a stream questions in an easy-to-answer format was downright *fun*. I found myself putting more free time into the exercises. After all, who doesn't love to do some math problems here and there? (It was a rhetorical question. The majority of you can put your hands down now.)

As expected, the exercises weren't challenging. As a math major, I was not only well-trained in mathematical thinking but I also used the principles of geometry and algebra constantly, so there was nothing there I was not familiar with. However, a handful of exercises used tricks (like converting repeating decimals into fractions) or identities (like trig identities) that didn't come to mind and prompted me to use the "I'd like a hint" button to see a solution strategy.

To make the exercises more engaging I restricted myself to not using scratch-work or a calculator whenever it was possible. This made some of the exercises more difficult and lead to various mistakes in some of exercises requiring multiple steps. It was a good challenge, though, and some of the exercises forced me to keep more objects in working memory than was comfortable. I actually had a few exercises where I constantly made mistakes due to bad mental work, typos, or hasty shortcuts (such not bothering to check whether 221/299 can be reduced by a factor of 13).

While I worked from start to finish fairly linearly, I didn't race through the material. I backed up and re-did exercises that I found particularly fun and re-did some exercises using slightly different methods. (Although I'd estimate only 1/5 of my total points came from re-doing exercises, I think they are worth fewer points once you are proficient at them.) I finished all 380 exercises in about 2 months of consistent work. Khan Academy is constantly adding content, so I plan to do future exercises they add.

## The Educational Experience

Although I couldn't see the exercises from the point of view of a student with little mastery of the content, I had some thoughts on their educational value.

I think that the biggest benefit of the Khan Academy exercises is that they can supply an unlimited number of practice problems and provide immediate feedback and explanations. This is something that's hard to do in a non-computer setting. A teacher can only spend time on so many examples during class and homework doesn't give a student immediate feedback. The strength of automation is that examples are infinite and feedback is instant and I think they leveraged both of those aspects well.

The tree of interconnected math concepts was well done. I wish I knew of a similar detailed tree for higher math topics. A very select few of the tree relationships did seem backwards, where I had a hard time believing that the a latter exercise offered any challenge a former exercise didn't. (I now wish I had saved examples, but I didn't.)

The exercises did a good job of focusing on one thing at a time. When learning any new skill it usually helps to isolate it in a familiar environment and focus on the unfamiliar part, so by focusing only on one idea per exercise these exercises would likely be a very helpful learning aid for someone who is trying to improve a specific weak point in their math skills. Combined with the tree as a whole, it would probably be easy to backtrack from an exercise full of confusing ideas to the lowest point of weakness and then work on refining the relevant skills from the bottom up. I can see a coach being able to do this easily for a student, or a motivated student doing it them self.

The system uses some clever math and machine learning to estimate when students are have achieved proficiency in an exercise (but all the student sees is a progress bar). Not knowing this when I began, it was still immediately clear that solving the first several problems in a set quickly and accurately earned proficiency but a couple of mistakes required numerous correct problems to earn proficiency. The system was well-tailored and let an initial burst of obvious competence pass quickly, while not passing mostly-right-but-still-struggling performance.

The "I'd like a hint" button was not the learning resource I hoped it would be. It offered step-by-step solutions (allowing the user to reveal one step of the solution at a time) but the steps were very formulaic without much explanation. For example, a typical step would read like "next we take the X and blah it with the Y", but with no mention of why this was necessary or possible. The exercises definitely were not teaching resources by themselves. Each problem did have a link to the associated teaching video, though, so they weren't designed to stand-alone.

Here are some of exercises that I thought were particularly interesting:

The group of triangle proofs. This set allows for a unique chance to walk through geometry proofs step by step. The way they set it up, each step was verified as you input it, so it was pretty much impossible to get it wrong. The geometry diagrams would light up the relevant portions each time you input a new step or hovered over an old step, so I think it would give nice feedback for students who have a harder time "seeing" the geometrical aspects of what they're doing.

The exercise on derivative intuition. A simple introductory concept that was well-illustrated. I'd recommend all beginning calculus students do this exercise. There were a couple other exercises oriented at building "intuition" for a topic. I really liked this, since I'm in favor of teaching intuitive concepts. (These should be replaced by rigor later on, of course, but many students, such as myself, benefit from these sorts of "gimme a clue or view as to what's happening here" explanations.)

The group of logical reasoning. A quick review of the basic steps of logic and the relevant syntax. This is typically covered in the first few weeks of a first-year introduction to math college course for math or computer science students. I'd recommend everyone in such a class go through the exercises to help cement those ideas.

Unfortunately, some of the exercises lacked diversity in the problem sets. As examples, when reducing fractions there were never common factors larger than 13 (possibly to keep it mentally doable, though), polynomials often seemed to be picked out of only a few sets of styles, and there were a limited number of triangle ratios used. Some of the problem sets were constrained by the intent of the problem, such as trying to pick a problem that resulted in clean answers, and those constraints left only a handful of possible starting values. Those were understandable, but some of them didn't have any such constraints. With problems that didn't vary much, a student could unconsciously get into a rote system of "take the number there, blah it against that number, blah the result and write it down". The entire point of Khan Academy is self-study so I'm not suggesting it needs to take anti-cheating measures, but there were some problems sets that seemed unnecessarily bland considering the pool of potential problems. With some of those exercises I would actually consistently get the exact same question twice within a span of eight questions.

There weren't many exercises on the deeper/more challenging topics. The practice exercises don't cover as much depth as the video lessons do and I would like to see more of the topics fleshed out. For example, there are very few Linear Algebra exercises yet many Linear Algebra videos. Calculus is limited to Differential Calculus only. Even in Algebra they covered a lot of specific skills, but left out some worthwhile ones. They're consistently adding new exercises, though, (they've added 13 exercises in the past three months alone) so hopefully these areas will be expended in the future.

## Miscellaneous Observations

On grading:

- You can achieve proficiency at any point during the exercise set, not just the end. You can keep an eye on the green star in the left-hand bar to see when you get it. Once achieved it can't be lost, although it can be recommended for review.
- I could almost always pass a problem set in four questions if I answered them correctly and within their "fast" time slot.
- The most efficient strategy that I found for passing a problem set was to get the first five questions right with three of them done at "fast" speed. Focusing on speed for all four would encourage hasty mistakes. Better to let the easy ones be fast and give them all good attention.
- Once a mistake was made, it would take at least 6, usually more, correct answers to pass the set. Wrong answers were weighted much more negatively than slow answers.
- A mistake seems to set your progress bar for the exercise down to at most half-filled.

On answers:

- Their scratch pad interface is a nice idea, but it's impractical to draw/write things with the mouse if you need to be fast. I just used a pen and sticky notes when I wanted scratch paper.
- Allowable answer formats varied somewhat inconsistently. Fractions almost always had to be in reduced form, but not always, rounding varied from 0 to two decimal places, and these were across exercises that had nothing to do with reducing fractions or rounding decimals. It was worth checking their little "acceptable format" indicator to see what types of input was allowed.
- The answer parsing was reasonably flexible. For example, when entering multiples of pi you could write "5pi" or "5 pi". It was flexible enough to not be annoying.

## Final Thoughts

The Khan Academy exercises are a good source of practice material. They are not, however, a learning resource and do not replace real homework assignments and feedback by teachers. But I think they're solid supplementary material. I also think they would help someone who was once comfortable with material and wants to re-learn it.

I think Khan Academy did a good job making the exercise format pain-free and enjoyable.