What's the point of taking a class?
- Build a lasting intuition for the key ideas.
- During the course, understand it enough to solve problems.
- After the course, enjoy it enough to revisit.
That's why I learn things. Non-goals are transcribing what a teacher says, or cramming only to forget everything. (Yeah, it's a game we play, but we're stepping off the treadmill and only cheating ourselves. Most subjects have useful insights buried somewhere.)
So, here's my strategy when studying:
If an idea clicks, write down the Aha! moment in language you'd use yourself.
If it doesn't, write down the Huh? moment. Move on and try again later (such as with the ADEPT method).
Keep it simple, like the KonMari method of organizing: Look at everything in your house. Does it spark joy? Keep what does, thank and donate what doesn't.
A simple study plan: Go through the material. Did it click? Write down what helped, otherwise look for a better explanation.
Actual Example: Stanford Machine Learning Course (Coursera)
My current learning project is the Machine Learning Class on Cousera. I've read a smattering of blog posts, the subject is growing, and after my friend asked me to join the class, I had to sign up. (It's great.)
Here's where I'm keeping my notes, Aha, and Huh moments:
Machine Learning Notes on Google Docs
This is one of the best learning experiences I can remember. A few examples:
An Aha moment for each prerequisite
For the major concepts the course depends on, I keep a 5-second summary in mind. This underlying concept, why does it exist? In plain English, what does it mean?
- Linear Algebra: spreadsheets for your equations. We "pour" data through various operations.
- Natural log: time needed to grow. Helps normalize widely varying numbers.
- e^x: models continuous growth, has a simple derivative.
- Gradient: direction of greatest change, helps optimize.
- Calculus -Art of breaking a system into steps. With the gradient, we can move in the best direction.
I reference these snippets as I encounter new formulas.
Huh moment: Need to clarify a Formula
There was a formula that I expected to be positive ("cost" should be positive), yet it had a negative sign out front. What gives?
It turns out I had forgotten a part of the derivation, where we expected the natural log to be negative. (This happens when we take the logarithm of numbers less than 1 — in other words, we are going "back in time" and shrinking.)
I would have preferred the equation written another way, and I made a note of this Huh? moment.
Huh moment: Why is it named that?
Early in the course, we define a "cost" function which tracks the difference between our predictions and the real value.
Why not call this difference something normal, like error?
It turns out "cost" is used because later in the course, we have items to minimize (like the number of variables in our model) which are not directly related to the error. The "cost" captures things outside the model, like the complexity we have. (If two models make equally accurate predictions, prefer the simpler one.)
Ah, "cost" can include fuzzier concepts. (I'd still prefer that laid out up-front.)
Aha moment: Summarize the Course in Plain English
As I go through the course, I have a plain-English definition in mind. What's it all about?
Machine Learning: Create models with Linear Algebra, then improve them with Calculus.
- Linear Algebra lets us use many (tens, hundreds, thousands) of variables in a "math spreadsheet".
- Calculus lets us improve our spreadsheet via feedback on how well it's working. Using functions like e^x, ln(x), x^2, etc. make it easy to take derivatives. Absolute value, if/then statements, etc. aren't easy to work with.
Now my thinking becomes: What types of predictive models can I make? If Linear Algebra can describe it, let's use it.
The Result: Notes You'll Actually Enjoy Reading
After the course is done, you're left with a set of notes that make sense to you: the Ahas, Huhs, and other gotchas. (This website is a running collection of mine.)
Future learning gets that much easier. Remember how you were confused about a topic a few years ago? Well, let's read the explanation you wrote to yourself on how to overcome it. Over time you build up a massive collection.
Other tips:
Embrace your confusion. The hesitation you feel when you see a formula is ok. Try to break down each part of the equation, ask what it means, make note of what is confusing and return over time. Every positive sign, every variable, why are they there?
It's ok to forget things - I do all the time. I just want a list of intuitions to load up when needed. Often a single phrase or diagram will bring it all back.
These notes are meant for you. Make them fast and quick. (My notes eventually become articles, but they stay informal and for my own use till then.)
The textbook already exists. Don't simply copy what the teacher/book said, add what you need to make it clear.
Show me your Aha! moments
This course is among the most fun I've had -- this is what learning should feel like, exploration with constant refinement. I'm curious to see if this approach helps you too.
For your next course, try keeping your notes in a single Google doc. Write down your Aha! and Huh? moments. Send me a link and I'll add them to this list:
- Kalid Azad - Coursera Machine Learning
- [you go here]
I'm curious to see what works for you, feedback is always welcome.
Happy math.
Other Posts In This Series
- Developing Your Intuition For Math
- Why Do We Learn Math?
- How to Develop a Mindset for Math
- Learning math? Think like a cartoonist.
- Math As Language: Understanding the Equals Sign
- Avoiding The Adjective Fallacy
- Finding Unity in the Math Wars
- Brevity Is Beautiful
- Learn Difficult Concepts with the ADEPT Method
- Intuition, Details and the Bow/Arrow Metaphor
- Learning To Learn: Intuition Isn't Optional
- Learning To Learn: Embrace Analogies
- Learning To Learn: Pencil, Then Ink
- Learning to Learn: Math Abstraction
- Learning Tip: Fix the Limiting Factor
- Honest and Realistic Guides for Learning
- Empathy-Driven Mathematics
- Studying a Course (Machine Learning) with the ADEPT Method
Leave a Reply
18 Comments on "Studying a Course (Machine Learning) with the ADEPT Method"
Great to see this article. Thanks alot…:-))))
Thanks Sajil!
Thanks Kalid! Very helpful and serves as a reference.
Just before the semester starts :). Good read and thanks!
Thanks for the article .. It is really good .. I have seen other studying methodologies as well like SQ3R (Survey Question, read, write and revise) and due to the absence of revision i tend to forget a lot of things.. But after reading this article , I guess my methodology is quite inaccurate.. Will implement this in future and check for myself .. thanks once again.
Thanks Raveesh. I’m not too particular on the details except for “write down what clicks, keep looking otherwise”. Hope that helps.
Thank you! Your notes give me a clear sense of what I need to learn. I most like the Goldilocks underfitting-overfitting-just right visual examples. The graphics are easy to understand.
Thanks Betsy! I like visual examples too, it helps me remember.
I really wish more people will try to learn this way, thus sharing knowledge this way. I realize recently how bad the overall communication is in STEM field: people are most of the time talking about steps and mechanics, scratching surfaces, using inappropriate and short-sighted analogies creating tons of confusions for the students. Go to most lessons on math on Youtube, even Sal Khan is most of the time showing you how to use it. Why everyone is talking about critical thinking but instead dumping facts and mechanics all the time? Why no one is inspiring good questions?
And by the way I have always wanting to take machine learning and AI. Reading your notes will definitely save me times :-)
Just recently I am reading on electricity/signals and I realized how inaccurate, crude most of our childhood days textbooks are in teaching science. When I was a kid, most of the stuff taught were in a “notional” or absolute amount, like pressure, current, velocity, while actually they are all “change” of something. I guess educators were in a dilemma to avoid talking about notion of change because that’s where calculus surface. Therefore lots of topics were taught in a awkward way unable to survive serious thinking. As William Beaty says, lots of things we learnt as a kid we will have to un-do them first in order to make room for completely new perspective.
I seriously think calculus should be introduced in the earlier years, and to be honest, if it is explained in the ADEPT fashion, why would it be difficult to understand?
Yep, I think Calculus can be learned much earlier (at least at an intuitive level) with more details provided over time.
Is like a commonplace book?
And you can put in this book an area of learning or learning math.
I’ve finished this course long time ago. And I also finished Intro to Machine Learning at Udacity.com and Deep Learning at that site too. They are fun and thought provoking.
What is the difference between machine learning and AI?
And by the way, statistical tools are really powerful and useful in AI machine learning, so powerful that a lot of times we might have techniques way ahead of any intuition. An example would be those algo running in Wall street accountable for 70% of the trades. I personally feel very uncomfortable with the lack of intuition in applying complicated models. Without understanding the assumptions and limitation of those data driven tool, they will become a Frankenstein one day, as evidenced by the more severe market volatility and Wall street storms.
Khalid, where do you keep track of your notes? Which program to annotate the images? I know there are so many, I never seem to use any. What’s your choice here? Btw, I took that course a month ago, and it’s an awesome intro! I also liked finding Octave, but more importantly, what you say here to be a ADEPT teacher.
I typically keep notes as text files on my computer, but moving to Google docs since it’s easier to add images. I’m using Skitch on OSX to annotate. I generally want the lowest-friction tools, if one gets cumbersome I use a simpler one (I want the smallest gap between having a thought and getting it recorded). Octave is really cool, glad I’m learning it :).