In this post, we address the importance of Linear Algebra in Machine Learning and discuss why we have to take it seriously!
Introduction
Our goal is to investigate the importance of Linear Algebra in Machine Learning. Let me start with one fact: The majority of us dislike math if not hate it! We try to avoid it at all costs and when we feel like we can’t avoid it, it’s like being in agony as our lungs blister with chemical burns. Am I right? We may have many different reasons, consciously or subconsciously:
- While reading it, it always looks like that we have to learn new things which we do not know how useful they are, except for understanding what we are reading!
- Why should I spend too much time one something that I don’t receive an immediate benefit? What if I can skip the math and still be able to do the job?
- Maybe due to the conformity bias, It looks cool to say, “I hate mathematics.”
- Maybe you had a bad teacher! I am not talking about being good or bad in actually knowing math; I am talking about a teacher that always discouraged you by criticizing your manner or questioning your ability! It usually happened. Right?
- Any other reason that you have in mind and I missed them …
All in all, the list is too long. Many times understanding the root of this problem may not solve the problem. Perhaps it’s good to only think about our motivations rather than forcing ourselves to like it! I personally believe there is no reason to like something if we have to do it. On the other hand, it’s actually a skill: Doing things we hate to do! So, let’s forget about liking math for a moment and try to just motivate ourselves to learn it at the level that we need!
A Fundamental Question
A question still remains: Do we really need to learn math if we desire to learn Machine Learning? The short answer is: (1) yes, you need to learn some branches of mathematics, such as Linear Algebra, (2) you need to learn it as needed. There is no need to be a master!
In this article, we focus on why we need Linear Algebra as a MUST known branch of mathematics. If you desire to be at least above average in knowing Machine Learning, YOU NEED IT.
Why Linear Algebra is Necessary for Machine Learning?
Linear Algebra is a branch of math that is broadly used everywhere. It is, in essence, the study of vectors and linear functions and we use it to interpret and utilize Machine Learning. Knowing Linear Algebra enlighten you about how algorithms work at their core level, which helps you to employ an algorithm better or even make a decision whether that choice (the utilized algorithm/approach) is practical or not. An example is the K-nearest neighbor algorithm which is known to be one of the simplest Machine Learning algorithms. Even for that, you still cannot avoid knowing a minimal level of Linear Algebra, if you would like to know how to implement it.
Vectors and matrices are everywhere, especially in Deep Learning. Loosely speaking, Deep Learning is technically Machine Learning using Neural Networks and Big Data. Neural Networks are made of matrices, and they mainly operate under the laws of Linear Algebra. Henceforth, a good knowledge of linear algebra seems to be necessary for understanding and utilizing Machine Learning and Deep Learning algorithms and architectures. Let’s not ignore what we can easily see: Linear Algebra is a MUST KNOWN for Machine Learning.
But wait! Is that enough to exclaim the importance of Linear Algebra? Of course not. I like to address some situations that you need to know Linear Algebra, SERIOUSLY:
- Working with vectors: Can you ignore vectors and their associated characteristics while working on Machine Learning? I think NOT. They are everywhere. The foundation of Machine Learning is based on vectors and matrices.
- Delving deep in an algorithm: You can’t possibly hit the root and gain a core understanding of Machine Learning algorithms without knowing Linear Algebra. You may say, “who cares about core understanding?” Yeah, you can ignore it, but what if your implementation is not working? What if you want to improve a method? What if you want to know more than average?! Then you need it so bad!!
- You want to teach what you know: Yes, that is very important. Assume you want to DO Machine Learning, whatever that “DOING” means. Eventually, you want to share your knowledge by any mean. Explaining to other folks, your manager, your students, and the list goes on. Of course, if you work in a company, it is unlikely that any of your managers care about “how your algorithm works and what is inside it?!” BUT, they care about the results. Assume you want to convince them something is NOT working. Then you need to explain in detail. If your managers neither care about why your algorithm works nor you can convince them by going into the details of Linear Algebra, then you MUST SIMPLY QUIT and chase your prosperity elsewhere!
Those lines where only simple examples to showcase the importance of Linear Algebra in Machine Learning. It is hard (if not impossible) to touch base on each and every aspect of Linear Algebra that are important in Machine Learning, in just ONE post. I am sure you will, but again, I suggest you to google “The importance of Linear Algebra in Machine Learning” to explore more!
It looks impossible to ignore, right? In the beginning, it may be like torture to force ourselves to dig deep into it. BUT, if you are audacious, what you will gain is amazing knowledge which facilitates your climbing the ladder of Machine Learning.
Summary
In this post, we investigated the importance of math and, in particular, linear algebra in machine learning. Given the fact that this blog aims to teach Machine Learning and Deep Learning concepts, it is worth to have a series of tutorials on key linear algebra prerequisites required for Machine Learning. In future posts, I will provide a series of articles to cover the basics of linear algebra.
P.S. Please share with me your thoughts by commenting below. I might be wrong in what I say, and I love to know when I am wrong. Furthermore, your questions might be my questions, as well. It’s always good to become better even if being the best is impossible in our belief system. So let’s help each other to become better.
Resources
Below you see a selection of supplementary material that may be helpful to explore the Linear Algebra further.
Books
- Mathematical Methods in the Physical Sciences by Mary L Boas, John Wiley, and Sons, 3rd Ed, 2006. [refer to the chapter associated with Linear Algebra]
- Introduction to Linear Algebra by Gilbert Strang, Wellesley-Cambridge Press, 5th Ed, 2016. [Link]
- A great book by University of California Davis [Link]
Online Materials
- Khan Academy has many great resources [Link]
- You can check Grant Sanderson YouTube videos, which you can reach through his site [YouTube channel, Website]
- Wikipedia usually comes to our rescue [Link]
- You can refer to this awesome online MIT course [Course Page]