Vectors and vector spaces
Vector spaces – a very natural first stop in anyone’s study of linear algebra. So what exactly are they? Roughly speaking, a vector space is a set of elements, called vectors, that can be added together and also scaled (by elements in a field, say \(\mathbb{R}\) or \(\mathbb{C}\)).
How do we say this mathematically? Well first we need our set of elements, which we will call \(V\), an addition operation which we denote \(+: V \times V \rightarrow V\) and a scalar multiplication which we denote \(\cdot: \mathbb{F} \times V \rightarrow V\) (if you wish, you can think of the field \(\mathbb{F}\) as \(\mathbb{R}\) or \(\mathbb{C}\)). With this in place, we define a vector space as follows:
Definition
Let \((V,+,\cdot)\) be a set equipped with an addition operation and a scalar multiplication (as above). Then \((V,+,\cdot)\) is a vector space if the following 8 axioms hold:
- Additive identity: There exists a vector \(a \in V\) such that for all other vectors \(v \in V\) we have \(a+v = v\). We usually denote this element by \(0\) and call it the zero element in \(V\).
- Additive inverses: For all \(v \in V\) there exists a vector \(w \in V\) such that \(v + w = 0\). We usually denote the inverse of \(v\) by \(-v\).
- Additive associativity: For any three vectors \(u,v,w \in V\) we have \((u+v)+w = u + (v+w)\).
- Additive commutativity: For any two vectors \(v,w \in V\) we have \(v + w = w + v\).
- Identity element in scalar multiplication: The unit element in the field, \(1 \in \mathbb{F}\), must satisfy \(1 \cdot v = v\) for all \(v \in V\).
- Compatibility of field multiplication and scalar multiplication: For any \(a,b \in \mathbb{F}\) and \(v \in V\) we have \((ab)\cdot v = a \cdot (b \cdot v)\). That is, you can multiply first in the field and then scalar multiply with the element of \(V\), or you can scalar multiply one at a time. Note: This looks like some sort of associativity condition, whilst strictly speaking it is not since \(a,b,v\) do not all belong to the same object, it can be remembered that way.
- Distributivity of scalar multiplication over field addition: For all \(a,b \in \mathbb{F}\) and \(v \in V\) we have \((a+b) \cdot v = a \cdot v + b \cdot v\).
- Distributivity of scalar multiplication over vector addition: For all \(a \in \mathbb{F}\) and \(v,w \in V\) we have \(a \cdot (v+w) = a \cdot v + a \cdot w\).
Note: If you are familiar with the language of groups and rings then the axioms can be remembered as follows. Axioms \(1 – 4\) say that \((V+)\) is an abelian group, and axioms \(5-8\) say that \(\cdot: \mathbb{F} \rightarrow \text{End}(V)\) is a ring homomorphism.
Note: In practice, it is often worthwhile to include an “axiom 0”. Namely to check that the addition and scalar multiplication really are maps from \(V \times V \rightarrow V\) and \(\mathbb{F} \times V \rightarrow V\) respectively.
Okay, that’s quite a long and technical definition. So let’s see some examples to get a feel for what these things look like.
Example 1
For our first example, \(V = \left\{ \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} : a_i \in \mathbb{F} \right\} \) where \(\mathbb{F}\) is your favourite field, i.e. \(V\) is the set of \(n \times 1\) column vectors with entries in the field \(\mathbb{F}\). Not surprisingly, this set is usually denoted \(\mathbb{F}^n\), we will also use this notation.
We need an addition operation \(+: \mathbb{F}^n \times \mathbb{F}^n \rightarrow \mathbb{F}^n\). There is a clear candidate, namely the addition given componentwise. That is, we define the addition on vectors as follows:
$$ \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} = \begin{pmatrix} a_1 + b_1 \\ \vdots \\ a_n+b_n \end{pmatrix}. $$
Next, we need a scalar multiplication \(\cdot: \mathbb{F} \times \mathbb{F}^n \rightarrow \mathbb{F}^n\), we choose this as follows:
$$\lambda \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} \lambda a_1 \\ \vdots \\ \lambda a_n \end{pmatrix} \text{ for any } \lambda \in \mathbb{F}.$$
Now for the fun part… we must check the 8 axioms… Expand the tab below to see the details. One final thing to notice is when \(n =1 \) our set is just the field \(\mathbb{F}\). Therefore a field is a vector space over itself.
Checking the axioms
Let us carefully check the axioms one by one.
- Let \(\underline{0} = \begin{pmatrix} 0 \\ \vdots \\ 0 \end{pmatrix}\) denote the vector in which all the entries are 0. We claim that \(\underline{0}\) is the zero vector. Indeed, for all \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \in V \) we have \(\begin{pmatrix} 0 \\ \vdots \\ 0 \end{pmatrix} + \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} 0+ a_1 \\ \vdots \\ 0+a_n \end{pmatrix} = \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \).
- Let \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \in V \). Then consider \(\begin{pmatrix} -a_1 \\ \vdots \\ -a_n \end{pmatrix} \in V\). We have \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \begin{pmatrix} -a_1 \\ \vdots \\ -a_n \end{pmatrix} = \begin{pmatrix} a_1-a_1 \\ \vdots \\ a_n-a_n \end{pmatrix} = \underline{0} \), so we have found an additive inverse.
- Let \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}, \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix}, \begin{pmatrix} c_1 \\ \vdots \\ c_n \end{pmatrix} \in V\). Then \(\left( \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix}\right) + \begin{pmatrix} c_1 \\ \vdots \\ c_n \end{pmatrix} = \begin{pmatrix} a_1+b_1+c_1 \\ \vdots \\ a_n+b_n+c_n \end{pmatrix} = \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \left(\begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} + \begin{pmatrix} c_1 \\ \vdots \\ c_n \end{pmatrix} \right) \) and so the addition is associative.
- Let \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}, \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} \in V\). Then \(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} = \begin{pmatrix} a_1+b_1 \\ \vdots \\ a_n+b_n \end{pmatrix} = \begin{pmatrix} b_1+a_1 \\ \vdots \\ b_n+a_n \end{pmatrix} = \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} + \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}\) so the additive is commutative.
- Let \(1 \in \mathbb{F}\) be the identity in the field. Then \(1 \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} 1 \cdot a_1 \\ \vdots \\ 1 \cdot a_n \end{pmatrix} = \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}\).
- Let \(\lambda, \mu \in \mathbb{F}, \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \in V\). Then \((\lambda \mu) \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} (\lambda \mu) a_1 \\ \vdots \\ (\lambda \mu) a_n \end{pmatrix} = \begin{pmatrix} \lambda (\mu a_1) \\ \vdots \\ \lambda (\mu a_n )\end{pmatrix} = \lambda \cdot \begin{pmatrix} \mu a_1 \\ \vdots \\ \mu a_n \end{pmatrix} = \lambda \cdot \left( \mu \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \right).\)
- Let \(\lambda, \mu \in \mathbb{F}, \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} \in V\). Then \((\lambda + \mu) \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} (\lambda + \mu) a_1 \\ \vdots \\ (\lambda + \mu) a_n \end{pmatrix} = \begin{pmatrix} \lambda a_1 + \mu a_1 \\ \vdots \\ \lambda a_n + \mu a_n \end{pmatrix} = \lambda \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \mu \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}\)
- We have \(\lambda \cdot \left(\begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} \right) = \lambda \cdot \begin{pmatrix} a_1 + b_1 \\ \vdots \\ a_n + b_n \end{pmatrix} = \begin{pmatrix} \lambda (a_1 + b_1) \\ \vdots \\ \lambda (a_n + b_n) \end{pmatrix} = \begin{pmatrix} \lambda a_1 + \lambda b_1 \\ \vdots \\ \lambda a_n + \lambda b_n \end{pmatrix}\) \(= \lambda \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} + \lambda \cdot \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix}\) for all \(\lambda \in \mathbb{F}, \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix}, \begin{pmatrix} b_1 \\ \vdots \\ b_n \end{pmatrix} \in V\).
Phew! All axioms have been checked and are satisfied. So \(\mathbb{F}^n\) is a vector space.
Note: When the addition and scalar multiplication are clear, one usually just writes the set to denote the vector space. So for example, you will see “Let \(\mathbb{F}^n\) be a vector space” much more commonly than you will see “Let \((\mathbb{F}^n,+,\cdot)\) be a vector space”.
Example 2
Now let \(V = \mathbb{F}[x]_{\leq 3}\) be the set of polynomials with coefficients in \(\mathbb{F}\) of degree less than or equal to \(3\). We define the addition of vectors to be the normal addition of polynomials, \((f+g)(x) = f(x) + g(x) \) and scalar multiplication to be the normal scalar multiplication of polynomials, \( (\lambda f)(x) = \lambda f(x)\). Once again the 8 axioms can be checked and are satisfied (the idea is very similar to that of example 1).
Just as important as examples are counterexamples. So let’s see a couple of those before moving on.
(Counter)example 3
Just as in example 1, let our set be \(V = \left\{ \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} : a_i \in \mathbb{F} \right\} \) with the addition of vectors defined componentwise (again, as in example 1), but now with scalar multiplication given by:
$$ \lambda \cdot \begin{pmatrix} a_1 \\ \vdots \\ a_n \end{pmatrix} = \begin{pmatrix} \lambda a_1 \\ \vdots \\ a_n \end{pmatrix}.$$
Now let’s check if all axioms are checked. Since the additive structure has not changed, axioms 1-4 are satisfied by our work done in example 1. As for the scalar multiplication, axioms 5,6 and 8 are satisfied. However, axiom 7 does not hold. We have
Since axiom 7 does not hold we do not have a vector space.
(Counter)example 4
Let the field be \(\mathbb{R}\). Consider the set \(V = \mathbb{Z}\), the integers, with the usual addition and scalar multiplication given by \(\lambda \cdot n = \lambda n\) for \(\lambda \in \mathbb{R}\) and \(n \in \mathbb{Z}\). Then by viewing the integers \(\mathbb{Z}\) as a subset of \(\mathbb{R}\), and writing down symbols without too much thought about where all the elements live, one could falsely convince themselves that the axioms 1-8 all hold.
Except there is a problem!
Our supposed scalar multiplication \(\mathbb{R} \times \mathbb{Z} \rightarrow \mathbb{Z}\) does not make sense, since for example we have \(\frac{1}{2} \cdot 1 = \frac{1}{2}\), which is not an integer. Therefore this is not an example of a vector space because our “axiom 0” doesn’t hold.
Subspaces
A common theme in mathematics when given an object (in this case a vector space) is to study its subobjects. The subobjects of vector spaces are the vector subspaces, or just subspaces for short.
Definition
A vector subspace of a vector space \(V\) is a non-empty subset \(W \subset V\) such that \(W\) equipped with the same addition and scalar multiplication as \(V\) is a vector space in its own right. (That is, forgetting about the fact that it lives inside \(V\)).
So, given a subset equipped with the addition and scalar multiplication from \(V\), how do we actually check if it is a subspace? We can of course check all the axioms (including, crucially, our axiom 0). However, there is a much quicker way! Namely, we can just check the following condition.
\(W \subset V\) is a subspace if and only if \(\lambda v + \mu w \in W\) for all \(\lambda, \mu \in \mathbb{F}\) whenever \(v, w \in W.\)
It is an exercise at the end of the section to check that this condition is sufficient. Let us see some examples of subspaces.
Example 5
For every vector space \(V\) the subsets \(\{0\} \subset V\) and \(V \subset V\) are both trivially subspaces of \(V\). The former is commonly referred to as the zero subspace.
Example 6
Consider the vector space \(\mathbb{F}^2 = \left\{ \begin{pmatrix} a \\ b \end{pmatrix} : a, b \in \mathbb{F} \right\}.\) Let \(W = \left\{\begin{pmatrix} a \\ 0 \end{pmatrix} : a \in \mathbb{F} \right\}.\) Let us see that \(W\) is a subspace of \(V\). To do this we just have to check the condition stated above holds.
Indeed, let \(\begin{pmatrix} a \\ 0 \end{pmatrix}, \begin{pmatrix} b \\ 0 \end{pmatrix} \in W\). Then for all \(\lambda, \mu \in \mathbb{F}\) we have \(\lambda \cdot \begin{pmatrix} a \\ 0 \end{pmatrix} + \mu \cdot \begin{pmatrix} b \\ 0 \end{pmatrix} = \begin{pmatrix} \lambda a + \mu b \\ 0 \end{pmatrix} \in W\). Therefore \(W\) is a subspace.
Summary and exercises
In this section, we have seen the definition of a vector space and some examples and non-examples of them. A vector space consists of a set \(V\) with an addition \(+\) and scalar multiplication \(\cdot\), however, we usually suppress the addition and scalar multiplication and just write \(V\) for the vector space. We also defined a vector subspace and stated an easy-to-check condition for a subset to be a subspace. We finished the section by seeing a couple of examples of subspaces.
Below are some exercises to test your understanding. Full solutions are given for all exercises. See you in the next section!
Exercise 1
Let \(V = \left\{ \begin{pmatrix} a & b \\ c & d \end{pmatrix} : a,b,c,d \in \mathbb{R} \right\}\) be the set of \(2 \times 2\) matrices over the real numbers. Define the addition of two matrices in the usual way, i.e.
Furthermore, define the scalar multiplication by
\(\lambda \cdot \begin{pmatrix} a & b \\ c & d \end{pmatrix} = \begin{pmatrix} \lambda a &\lambda b \\ \lambda c &\lambda d \end{pmatrix}\)
Show that \(V\) with this addition and scalar multiplication is a vector space.
Solution
The 8 axioms need to be checked. The solution is very similar to the workings for example 1, so have a look back there for inspiration if needed.
Exercise 2
Let \(V\) be a vector space. Using the axioms, show that the zero vector \(0 \in V\) is unique. Show also that for any \(v \in V\) the additive inverse \(-v\) is unique.
Solution
In reality, this is more of an exercise in group theory. However, it is a good result to know so feels worthwhile to include here.
Suppose \(0\) and \(0’\) are two zero vectors. Then since \(0\) is a zero vector we have \(0 + 0′ = 0’\). Similarly, since \(0’\) is a zero vector we have \(0′ + 0 = 0\). Finally, since the addition of vectors is commutative we have \(0 + 0′ = 0\). Combining this we have \(0 = 0 + 0′ = 0’\).
The proof that the additive inverse is unique is very similar. Let \(v \in V\) and suppose \((-v) \in V\) and \(w \in V\) are both additive inverses for \(v\). Since \((-v) \in V\) is an additive inverse we have \(v + (-v) = 0\). Adding \(w\) to both sides gives \(v + (-v) + w = w\). Finally using the commutativity and the fact that \(v + w = 0\) we have \(-v = w\), as required.
Exercise 3
Let \(V\) be a vector space. Suppose that \(U\) and \(W\) are subspaces. Show that \(U \cap W\), the intersection of \(U\) and \(W\), is also a subspace of \(V\).
Solution
We must show that if \(u \in U \cap W\) and \(v \in U \cap W\) then for all scalars \(\lambda, \mu \in \mathbb{F}\) we have \(\lambda u + \mu v \in U \cap W\).
Indeed, since \(u,v\) are in the intersection we have \(u,v \in U\) and so \(\lambda u + \mu v \in U\).
Similarly, we have \(u,v \in W \) and so \(\lambda u + \mu v \in W\). Therefore \(\lambda u + \mu v\) lies in the intersection.
Exercise 4
Let \(V = \mathbb{R}^3\). Let \(a,b,c \in \mathbb{R}\) and set \(W = \left\{ \begin{pmatrix} x \\ y \\ z \end{pmatrix}: ax + by + cz = 0 \right\}\). Equip \(W\) with the same addition and scalar multiplication as \(V\). Show that \(W\) is a subspace of \(V\).
Optional: Take various values of \(a,b,c \in \mathbb{R}\) and plot it on desmos.
Solution
Let \(\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix}, \begin{pmatrix} y_1 \\ y_2 \\ y_3 \end{pmatrix} \in W\). Let \(\lambda, \mu \in \mathbb{F}\). Then we have \(\lambda \cdot \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} + \mu \cdot \begin{pmatrix} y_1 \\ y_2 \\ y_3 \end{pmatrix} = \begin{pmatrix} \lambda x_1 + \mu y_1 \\ \lambda x_2 + \mu y_2 \\ \lambda x_3 + \mu y_3 \end{pmatrix}\).
Now consider \(a(\lambda x_1 + \mu y_1) + b(\lambda x_2 + \mu y_2) + c(\lambda x_3 + \mu y_3)\).
This equals \((a \lambda x_1 + b \lambda x_2 + c \lambda x_3) + (a \mu y_1 + b \mu y_2 + c \mu y_3) = \lambda (a x_1 + b x_2 + c x_3) + \mu (a y_1\) \(+ b y_2 + c y_3) = \lambda (0) + \mu (0) = 0\), and so \(W\) is a subspace.
On desmos, choosing values for \(a,b,c\) with at least one of them non-zero will give you a plane through the origin. For example, plotting \(2x + 3y + 6z = 0\) gives:
Exercise 5
Prove the if and only if condition we stated for a non-empty subset \(W \subset V\) to be a subspace. That is, prove that:
\(W \subset V\) is a subspace if and only if \(\lambda v + \mu w \in W\) for all \(\lambda, \mu \in \mathbb{F}\) whenever \(v, w \in W.\)
Solution
Our decision to explicitly think about an axiom 0 really pays off when it comes to subspaces – because, in fact, it is one of the only things that needs to be checked.
Let’s prove the only if direction first, i.e. prove that the left-hand side implies the right-hand side.
Suppose \(W \subset V\) is a subspace. Then by definition axiom 0 holds. That is, the addition is a map \(+: W \times W \rightarrow W\) and the scalar multiplication is a map \(\cdot: \mathbb{F} \times W \rightarrow W\). Thus, if \(v,w \in W,\) then \(\lambda v, \mu w \in W\) and also \(\lambda v + \mu w \in W\).
Now we show the backwards implication, i.e. show that the right-hand side implies the left-hand side. So we assume that if \(v,w \in W\) then \(\lambda v + \mu w \in W\) for all scalars \(\lambda, \mu\). We must check all the axioms (including axiom 0!!).
- Recall that axiom 0 requires us to check that the addition and scalar multiplication really are maps from \(W \times W \rightarrow W\) and \(\mathbb{F} \times W \rightarrow W\) respectively. Let \(v,w \in W\), then by our assumption \(v + w \in W\) so the addition is OK. Next, let \(w \in W\) and \(\lambda \in \mathbb{F}\), by axiom 1 (see below) we know that the zero vector \(0 \in W\). By assumption \(\lambda w + 0 = \lambda w \in W\) and so the scalar multiplication also satisfies axiom 0.
- Choose any \(w \in W\). Since \(W\) is non-empty this can be done. Then by assumption, we have \(w + (-1)w = 0 \in W\) and so \(W\) contains the zero vector.
- Let \(w \in W\). By axiom 1 we have \(0 \in W\). Therefore we have \(0 + (-1)w = -w \in W\). Therefore \(W\) contains additive inverses.
- For 3. and 4. we can utilise the fact that \(W\) is a subset of \(V\) with the same addition. Namely, if associativity holds for all elements of \(V\), then it certainly holds for any subset of elements of \(V\). Therefore associativity holds in \(W\) automatically.
- For the same reasons explained in 3., commutativity holds in \(W\) automatically.
- Similarly, we can make full use of the fact that \(W\) has the same scalar multiplication as \(V\). Since \(1 \in \mathbb{F}\) satisfies \(1 \cdot v = v\) for all vectors \(v \in V\). It must be the case that \(1 \cdot w = w\) for all \(w \in W \subset V\).
- Compatibility holds for \(V\), therefore it must hold for \(W\).
- Distributivity holds for \(V\), therefore it must hold for \(W\).
- See 7.