This pairs really well with the little book https://github.com/little-book-of/linear-algebra, I have recently updated it with more content and clearer explanations, so if you're diving into this topic, you might find it helpful.
Thank you for creating and open sourcing this. I have a question if I may.
If a person has the goal of getting into a field like computer vision or machine learning, would they be able to build useful things right away if they completed this book?
Definitely! If you scroll down a bit, you will see in Chapter 10 that I've included some fun applications, things like 2D/3D geometry, linear regression, recommender systems, and even a quick intro to PageRank. I wanted to show how these ideas connect to real-world problems, so it's not just theory. Hope you find it interesting.
if you know how to handle real or complex coordinate vectors and matrices you’re one only isomorphism away (aka choice of basis) from dealing with an “abstract” vector space (except if you want to talk about finite fields or infinite dimensions). it seems like a really good starting point for many learners’ backgrounds…
>f you know how to handle real or complex coordinate vectors and matrices you’re one only isomorphism away (aka choice of basis) from dealing with an “abstract” vector space
No, you aren't. How would you explain that matrices are both linear transformations and vectors? How would explain what a dual space is? How would you understand the properties of the Fourier transformation, which is a mapping between functions, which are also vectors, and itself also is a vector?
as i said if you want infinite dimensional things like Fourier transforms acting on function spaces you may benefit from additional abstraction. but even those people will benefit from having learned Rn first.
i’m as much with bourbaki as the next guy. but that’s not really how most engineers learn in practice.
as for treating linear maps between finite dimensional spaces as vectors that’s quite straightforward to do in coordinate terms.
again i refer you to the classics like Golub and van Lean that have been reprinted many many times and educated generations.
This is awesome, been going through the little book as a way to refresh myself on linear algebra which I see has been updated as well. Luckily only on chapter 4 so definitely going to start doing the labs alongside the book now!
You are quoting the lab which is all about doing stuff in numpy. Probably if you want to critique the definitions used you should switch to the book.
That said, it doesn't lead with a notion of vector that is as general as I'd like. Readers might be later surprised to find that there are vectors which are not lists of numbers. But I only looked at the first few pages so I assume the vector space axioms show up eventually.
I find it very disrespectful towards a project with such popularity to contributors ratio.
It takes long hours to produce a working notebook, a synergy between concept descriptions and theorical explanations alongside code, comments of code and markdown for the theory here and there is an artistic coupled with engineering stunt you should try to accomplish yourself before criticizing. Perhaps you have, in which case a contribution as the other comment points out is the courteous thing to do.
That book must have taken months to author and is one of the best read on the subject I've ever come across.
Edit: removed incorrect fact about scalars.. they don't have a direction they have polarity.
I tried to build intuition before going into the formal definitions.
Personally, I've always liked Bourbaki's books, but they're too formal for learning - especially in linear algebra, which I see as something meant for applications rather than pure math research.
Maybe I just oversimplified things or made them feel less "math".
Readers might be later surprised to find that there are vectors which are not lists of numbers
I see this sort of thing as being similar to how physics is taught. Year 1: Atoms are indivisible. Year 2: Well, no, actually, we lied, they consist of elementary particles called electrons, protons, and neutrons. Year 3: Well, technically the protons and neutrons aren't indivisible either. ... Year 10: OK, fine, we have no idea. Your turn, help us figure this out.
Nobody starts rambling about quarks and gluons in grade school, and few practitioners will ever need to deal with them at all. Likewise, for most people looking to get their feet wet in ML, vectors are a 1D list of numbers, matrices are a 2D list of numbers, and tensors are lists of numbers with any number of dimensions. Definitions that are incomplete at best, but good enough to get started.
And precisely this is the risk of teaching "mathematics" using a programming language. Unfortunately, I think quite a few people don't actually know the abstract mathematical definitions and use concrete implementations as definitions in their heads in the first place.
At least using a theorem prover would get you closer to doing actual maths and proving.
It's probably more accurate to say courses like these teach mechanics/calculation more than they teach the theory.
There are already an uncountable number of linear algebra textbooks on the market. This is not meant to be a formal math textbook, but is instead aimed at those looking to pick up practical application-level insights and skills. (The "Why this matters" sections in the book are a worthy improvement on the state of the learning art by themselves, IMHO.)
This collection of lab exercises seems far better suited to the purpose than the abstract proof-based resources that some of you seem to have in mind. Fortunately, those resources are still available to those who want to go further into the theory.
And if you feel you have something to add to linear-algebra pedagogy yourself, well... what's stopping you?
Marimo notebook would fit nicely here for interactive examples where you could play with values and see computations in real time, though it probably lacks in formatting capabilities and quite heavy.
The linear combination picture is extremely misleading: the gray dots show the linear combinations where the coefficients are integers, but we are in the vector space R^2! This could have been a great opportunity to teach that any vector in R^2 could become a linear combination of these two vectors, thus it could become a new coordinate system.
Even though this is probably introduced in a later chapter, a curious reader will be able to question why the gray dots are form with integer coefficients and leave them with a wrong impression.
Your comment would be just fine without that first sentence. When people share their work, they're putting themselves in a vulnerable position, and there's no need to lead with a smack.
That book is about NumPy and not linear algebra. For actually learning linear algebra you need a math book, pen and paper.
NumPy is the worst way of abstracting linear algebra into a programming language. Octave is better in not obfuscating the concepts with OO boilerplate and having actually useful printed matrix output instead of ugly list representations. Even Fortran is better.
In practice, you'll often encounter the code before the underlying math (especially in Machine Learning), so developing intuition from both directions is always helpful.
Btw, if you enjoy pure mathematics, starting from Chapter 4 in the book, I begin defining vectors and the axioms of vector spaces:
If you are looking for Python notebook that is beginner friendly, I'm working on desktop app called MLJAR Studio. It packs Python and JupyterLab in single executable - installation is stright-forward. Additionally, there are extensions that makes working with Python much easier for beginners: for example, variable inspector and packages manager.
This pairs really well with the little book https://github.com/little-book-of/linear-algebra, I have recently updated it with more content and clearer explanations, so if you're diving into this topic, you might find it helpful.
pairs well? Isn't it the same book?
Based on the URL correlation and content, it sure appears to be the same book.
By pairing, I mean that you can read the book alongside the notebook. Sometimes, in the notebook, I don't explain the concepts, only some Python code.
Thank you for creating and open sourcing this. I have a question if I may.
If a person has the goal of getting into a field like computer vision or machine learning, would they be able to build useful things right away if they completed this book?
Definitely! If you scroll down a bit, you will see in Chapter 10 that I've included some fun applications, things like 2D/3D geometry, linear regression, recommender systems, and even a quick intro to PageRank. I wanted to show how these ideas connect to real-world problems, so it's not just theory. Hope you find it interesting.
Much appreciated! I'll give this a go and see how far I get.
great work!, will take a look over the weekend.
Here is the Google Colab link for you https://colab.research.google.com/github/little-book-of/line...
>the core ideas of linear algebra
I think it is actually delusional to say that a book which does not properly define what a vector is, contains the "core ideas" of linear algebra.
Linear algebra is so much more than lines in R^n. It is a powerful theory because it is abstract.
if you know how to handle real or complex coordinate vectors and matrices you’re one only isomorphism away (aka choice of basis) from dealing with an “abstract” vector space (except if you want to talk about finite fields or infinite dimensions). it seems like a really good starting point for many learners’ backgrounds…
>f you know how to handle real or complex coordinate vectors and matrices you’re one only isomorphism away (aka choice of basis) from dealing with an “abstract” vector space
No, you aren't. How would you explain that matrices are both linear transformations and vectors? How would explain what a dual space is? How would you understand the properties of the Fourier transformation, which is a mapping between functions, which are also vectors, and itself also is a vector?
as i said if you want infinite dimensional things like Fourier transforms acting on function spaces you may benefit from additional abstraction. but even those people will benefit from having learned Rn first.
i’m as much with bourbaki as the next guy. but that’s not really how most engineers learn in practice.
as for treating linear maps between finite dimensional spaces as vectors that’s quite straightforward to do in coordinate terms.
again i refer you to the classics like Golub and van Lean that have been reprinted many many times and educated generations.
even Golub and van Loan basically start there.
Offer a better alternative, then? E-books are handy.
A vector is an ordered collection of scalars, arranged either in a row or a column
Seems like a valid definition of a vector to me. What should be added or removed, in your view?
This is awesome, been going through the little book as a way to refresh myself on linear algebra which I see has been updated as well. Luckily only on chapter 4 so definitely going to start doing the labs alongside the book now!
Nice to see it!
I think that a lot of mathematics would benefit from executable code (or I would say - executable maths) - so people can play and experiment with it.
I had my own approach for teaching deep learning (https://github.com/stared/thinking-in-tensors-writing-in-pyt...), showing both mathematical formulation in LaTex, and a numeric one in PyTorch.
would be great if upgrades to pytorch 2
What is a point of Linear Algebra Tutorial which does not even attempt to properly define what a vector is?
>A vector has:
>A symbol (we might call it v, w, or even →AB in geometry).
>Components (the individual numbers, like 2 and 3 in [2, 3]).
>An arrow picture (a geometric way to see the vector as a directed line segment).
None of these are properties of all vectors. They are properties of how vectors are represented in numpy.
You are quoting the lab which is all about doing stuff in numpy. Probably if you want to critique the definitions used you should switch to the book.
That said, it doesn't lead with a notion of vector that is as general as I'd like. Readers might be later surprised to find that there are vectors which are not lists of numbers. But I only looked at the first few pages so I assume the vector space axioms show up eventually.
I find it very disrespectful towards a project with such popularity to contributors ratio.
It takes long hours to produce a working notebook, a synergy between concept descriptions and theorical explanations alongside code, comments of code and markdown for the theory here and there is an artistic coupled with engineering stunt you should try to accomplish yourself before criticizing. Perhaps you have, in which case a contribution as the other comment points out is the courteous thing to do.
That book must have taken months to author and is one of the best read on the subject I've ever come across.
Edit: removed incorrect fact about scalars.. they don't have a direction they have polarity.
My intention was to suggest that my parent shouldn't be so quick to criticize while acknowledging that they may have a point.
If the latter goal outshined the former, my apologies. I love stuff like this.
The book is even worse: "A vector, by contrast, is an ordered collection of numbers."
This is a terrible definition and makes talking about proper linear algebra proper impossible.
I tried to build intuition before going into the formal definitions.
Personally, I've always liked Bourbaki's books, but they're too formal for learning - especially in linear algebra, which I see as something meant for applications rather than pure math research.
Maybe I just oversimplified things or made them feel less "math".
here you go mate, exactly what you are looking for : https://github.com/little-book-of/linear-algebra#contributin...
Every contribution is welcome: whether it's criticism, fixing typos, or even completely rewriting a section!
Writing this book has been a real challenge for me, since all the books I enjoy reading are very formal ones :))
Anyone here into Grothendieck's SGA or EGA? (Sorry, a bit off topic!)
Readers might be later surprised to find that there are vectors which are not lists of numbers
I see this sort of thing as being similar to how physics is taught. Year 1: Atoms are indivisible. Year 2: Well, no, actually, we lied, they consist of elementary particles called electrons, protons, and neutrons. Year 3: Well, technically the protons and neutrons aren't indivisible either. ... Year 10: OK, fine, we have no idea. Your turn, help us figure this out.
Nobody starts rambling about quarks and gluons in grade school, and few practitioners will ever need to deal with them at all. Likewise, for most people looking to get their feet wet in ML, vectors are a 1D list of numbers, matrices are a 2D list of numbers, and tensors are lists of numbers with any number of dimensions. Definitions that are incomplete at best, but good enough to get started.
What is the point of a comment saying something doesn’t properly define something that doesn’t share the proper definition?
(Posted in good fun, no harm intended)
And precisely this is the risk of teaching "mathematics" using a programming language. Unfortunately, I think quite a few people don't actually know the abstract mathematical definitions and use concrete implementations as definitions in their heads in the first place.
At least using a theorem prover would get you closer to doing actual maths and proving.
It's probably more accurate to say courses like these teach mechanics/calculation more than they teach the theory.
There are already an uncountable number of linear algebra textbooks on the market. This is not meant to be a formal math textbook, but is instead aimed at those looking to pick up practical application-level insights and skills. (The "Why this matters" sections in the book are a worthy improvement on the state of the learning art by themselves, IMHO.)
This collection of lab exercises seems far better suited to the purpose than the abstract proof-based resources that some of you seem to have in mind. Fortunately, those resources are still available to those who want to go further into the theory.
And if you feel you have something to add to linear-algebra pedagogy yourself, well... what's stopping you?
Marimo notebook would fit nicely here for interactive examples where you could play with values and see computations in real time, though it probably lacks in formatting capabilities and quite heavy.
The linear combination picture is extremely misleading: the gray dots show the linear combinations where the coefficients are integers, but we are in the vector space R^2! This could have been a great opportunity to teach that any vector in R^2 could become a linear combination of these two vectors, thus it could become a new coordinate system.
Even though this is probably introduced in a later chapter, a curious reader will be able to question why the gray dots are form with integer coefficients and leave them with a wrong impression.
So sad that you didn't like the book : (
I'm not really a visual guy, so let me rewatch the awesome 3Blue1Brown series and try to visualize it again:
https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
> I'm only halfway into the first chapter but I already hate it.
Can you please make your substantive points more respectfully? This is in the Show HN guidelines: https://news.ycombinator.com/newsguidelines.html.
Your comment would be just fine without that first sentence. When people share their work, they're putting themselves in a vulnerable position, and there's no need to lead with a smack.
Sorry! It's too late to edit my comment now but as moderator please feel free to delete that sentence.
Ok! I've done that and will mark my GP comment offtopic and collapse it.
That book is about NumPy and not linear algebra. For actually learning linear algebra you need a math book, pen and paper.
NumPy is the worst way of abstracting linear algebra into a programming language. Octave is better in not obfuscating the concepts with OO boilerplate and having actually useful printed matrix output instead of ugly list representations. Even Fortran is better.
Of course, this is about Python and NumPy.
In practice, you'll often encounter the code before the underlying math (especially in Machine Learning), so developing intuition from both directions is always helpful.
Btw, if you enjoy pure mathematics, starting from Chapter 4 in the book, I begin defining vectors and the axioms of vector spaces:
https://little-book-of.github.io/linear-algebra/books/en-US/...
hard disagree. octave and matlab (and maybe even r) are first and foremost calculator languages, not programming languages.
numpy is an intuitive way of doing those numerical calculations in a general use programming language. only minus is the missing gong and why commands
The best is of course Wolfram + the Mathematica libraries. You get full symbolic manipulation as well, so you can do abstract linear algebra as well.
This is great. Covers a lot of useful topics. Thanks..
If you are looking for Python notebook that is beginner friendly, I'm working on desktop app called MLJAR Studio. It packs Python and JupyterLab in single executable - installation is stright-forward. Additionally, there are extensions that makes working with Python much easier for beginners: for example, variable inspector and packages manager.
how is it different from Spyder?
That is awesome work. I really find it usefull.