What are basis vectors? Why are they so important?
What is Span and a Linear Combination?

If you read my last post, Linear Algebra Basics 1, where I introduced vectors, vector additions and scalars, then you are ready to continue with this post. In this post we will focus on scalars and how we can use them.


Let's expand upon the term scalar. What if each vector coordinate has a scalar? What do I mean by this?

Any vector has vector coordinates, and this vector has basis vectors. These basis vectors, $\hat{i}$ and $\hat{j}$ (i hat and j hat), are scalars. $\hat{i}$ goes along the x-axis and $\hat{j}$ goes along the y-axis. Usually we expect these basis vectors to both be 1, but we can just change them. Conceptually, we add a basis vector for each dimension, so, if we go 3-dimensional, we have an extra basis vector $\hat{k}$.

We scale the vector coordinates with the basis vectors like this:

$$ \vec{v} = \begin{bmatrix} 2(\hat{i})\\ 5(\hat{j}) \end{bmatrix} $$

Then if we assume our basis vectors to be standard, the equations would just be $2*1$ and $5*1$. So if we have a vector

$$ \vec{v} = \begin{bmatrix} 2\\ 5 \end{bmatrix} $$

We could scale it by multiplying with 3 on both vector coordinates (notice how easy of an operation it is to scale a vector):

$$ \vec{v} = 3\begin{bmatrix} 2\\ 5 \end{bmatrix} = \begin{bmatrix} 3(2)\\ 3(5) \end{bmatrix} = \begin{bmatrix} 6\\ 15 \end{bmatrix} $$

Basis vectors enables us to reach any point. The two exceptions to this is if we have all vectors are stuck at origin or on the exact same line, meaning all vectors are facing the exact same or opposite way. Here is a scenario, the left is 2 vectors with same vector coordinates stuck at the origin, while on the right, we have 2 vectors on the same line:

If we have 2 vectors, that both are not at the origin and not on the same line. This means that we can reach ANY point in a given dimension. If we have two vectors and calculate a third vector with vector addition, then that third vector can reach any point, if we vary $\hat{i}$ and $\hat{j}$ of the first and second vector. Here is an example, where we simply change $\vec{v}$'s and $\vec{u}$'s basis vectors to the exact opposite. Notice that the basis vectors could be any real numbers, and therefore we can freely reach every possible point in the 2-dimensional plane:

opposite basis vectors

Now, after understanding scalars and basis vectors, it is the perfect time to introduce span and linear combinations, as they are closely related.

I just showed you that we can add vectors together, $\vec{v}$ and $\vec{u}$; the third resulting vector would be $\vec{w}$. Then if we add $\vec{v}$ and $\vec{u}$ together, we call it a linear combination of the two vectors. If you take those two vectors, $\vec{v}$ and $\vec{u}$, and find every possible combination, then we call that the set of all possible linear combinations, which is also called the span. Here is a definition of span:

span is all the linear combinations of $\vec{v}$ and $\vec{u}$, where $\vec{v}$ and $\vec{u}$ has basis vectors, $\hat{i}$ and $\hat{j}$, that vary over all real numbers

This means we have $a\vec{v}$ and $b\vec{u}$, where a and b represents $\hat{i}$ and $\hat{j}$. A linear combination of three vectors or more dimensions is certainly possible, and the notation is the same. We always have a basis vector for each vector.

The next and last terminology I want to introduce is Linear Dependence and Linear Independence. If we can remove a vector from our set of vectors, without reducing the span, we call that set of vectors linearly dependent. This is usually in the case of the set of vectors line up or is stuck at the origin. But if we cannot remove a vector from our set of vectors, without reducing the span of that set of vectors, we call that set linearly independent.

The set of vectors on the left is linearly dependent, because they line up, and therefore, if we remove one of the vectors, we will still have the same span as before. The set of vectors on the right is lineraly independent, because they do not line up, and therefore, if we remove a vector, we would be reducing the span.

If you were asked the question, what is the span of these vectors? You could really ask yourself, if we were to use vector addition and scaling, what are all the possible points we could reach? In 2-dimensional cases, it is visually quite clear which points we can reach, but it would probably be quite hard to imagine 10-dimensional space. In any case, you really want to find the points which you can't reach.


Summary (of the questions at the top):

  1. What are basis vectors? Why are they so important?
    A vector has vector coordinates. But a vector also has basis vectors for each vector coordinate. They are also referred to as i hat, $\hat{i}$, which we assume goes 1 along the x-axis, and j hat, $\hat{j}$, which we assume goes 1 along the y-axis in a coordinate system.
  2. What is Span and Linear Combinations?
    Span contains a set of vectors. If you do combinations of those vectors in the set, e.g. performing vector addition or scaling, you get a linear combinations. Then if we find all linear combinations, we call that the span. There is no method or formular for finding all the possible linear combinations, but rather you would find the linear combinations that you cannot include in your span. If you find those that we exclude, we could say the span is all linear combinations except blank or the span is all linear combinations except the one's within the range of blank.