I have an unreasonable love for linear algebra. It might be because of my great professors, or maybe the fact that it’s a subject that I can visualize and make sense of, but either way, there is something super satisfying about learning why something works the way it does.

A few months ago, I was writing a program that involved computing the angle between two vectors. The program was going to do this a lot (I think roughly 150 million times) with different vectors. I made the (poor) decision to use Python for a compute-heavy task. To make matters worse, this was back before I knew the wonders of Numba.

To start this project, I did what any good programmer does. I googled how people find the angle between two vectors. The industry standard way of finding the angle between two vectors is a formula that relies on the dot product of two vectors.

The formula looks like this:

$$\theta = arccos \bigg( \frac{a \cdot b}{|a| |b|} \bigg)$$

Wait, what’s a dot product?⌗

It’s that little dot between a and b, as shown here: $a \cdot b$

A dot product is when you take two vectors, multiply each vector’s components $\langle \hat{i}, \hat{j}, \hat{k} \rangle$ (think x, y, z) together, and get the sum of the products.

So the dot product of $$ \begin{bmatrix} 2\\ 7 \end{bmatrix} \cdot \begin{bmatrix} -3\\ 1 \end{bmatrix}$$ is calculated like this: $$ =(2*-3) + (7 * 1) = -6 + 7 = 1 $$

The Problem⌗

Computing the angle with that formula was really slow. It took nearly 40 minutes to compute, and I had a time limit of 20 minutes.

Delving Deeper⌗

Given two vectors, I want to get the angle between them. Think of this is as a trig problem. Given two sides of a triangle, find the angle that those two sides form with eachother.

Small problem. If you remember from trig, you need at least 3 pieces of information to calculate the missing angle. It could be two sides and an angle, three sides, or one side and two angles. No matter how you do it, you need three pieces of information¹.

Sadly, we can’t just plug both sides into a trig function and call it a day. We need to perform some operation on the vectors to get an additional piece of information. We could figure out the length of the vector C (from vector A to vector B), or we could determine one of the other angles.

Calculating Side Length⌗

Calculating side length is the same as calculating the magnitude of a vector. For that, we can just use the vectors components and plug them into the Pythagorean theorem.

$$|v|=\sqrt{{v_\hat{i}}^2+{v_\hat{j}}^2+…+{v_\hat{k}}^2} $$

Great! Now we can get the side length of any vector.

Method #1: Dot Product⌗

Here’s the formula from earlier: $$\theta = arccos \bigg( \frac{a \cdot b}{|a| |b|} \bigg)$$

We will derive that using the Law of Cosines².

If it’s been a while since you’ve taken geometry, you may have forgotten about the law of cosines. It allows you to calculate an angle using only the side lengths of a triangle.

$$c^2 = a^2 + b^2 − 2ab \cos (\theta)$$

Translating this to in terms of our vectors.

$${|v-u|}^2 = {|v|}^2 + {|u|}^2 − 2{|v|}{|u|} \cos (\theta)$$

Now, let’s expand the expression using the side length formula we defined earlier.

$$\bigg({\sqrt{{(v_\hat{i} - u_\hat{i})}^2+{(v_\hat{j} - u_\hat{j})}^2}\bigg)}^2 = {\bigg(\sqrt{{v_\hat{i}}^2 + {v_\hat{j}}^2}\bigg)}^2 + {\bigg(\sqrt{{u_\hat{i}}^2 + {u_\hat{j}}^2}\bigg)}^2 − 2{|v|}{|u|} \cos (\theta)$$

$${(v_\hat{i} - u_\hat{i})}^2+{(v_\hat{j} - u_\hat{j})}^2 = {{v_\hat{i}}^2 + {v_\hat{j}}^2} + {u_\hat{i}}^2 + {u_\hat{j}}^2 − 2{|v|}{|u|} \cos (\theta)$$

$${v_\hat{i}}^2 + -2{(v_\hat{i}*u_\hat{i})} + {u_\hat{i}}^2 + {v_\hat{j}}^2 + -2{(v_\hat{j}*u_\hat{j})} + {u_\hat{j}}^2 = {{v_\hat{i}}^2 + {v_\hat{j}}^2} + {u_\hat{i}}^2 + {u_\hat{j}}^2 − 2{|v|}{|u|} \cos (\theta)$$

Remove the duplicate ${u_{i}}^2$ and ${v_{i}}^2$ terms from each side:

$$-2{(v_\hat{i}*u_\hat{i})} -2{(v_\hat{j}*u_\hat{j})} = − 2{|v|}{|u|} \cos (\theta)$$

$${(v_\hat{i}*u_\hat{i})} + {(v_\hat{j}*u_\hat{j})} = {|v|}{|u|} \cos (\theta)$$

$$ v \cdot u = {|v|}{|u|} \cos (\theta)$$

$$ \frac {v \cdot u}{{|v|}{|u|}} = \cos (\theta)$$

$$ \theta = \arccos {\bigg(\frac {v \cdot u}{{|v|}{|u|}}\bigg)} $$

Great! Now we know how the angle relates to the dot product of two vectors.

Also note that if $|u|=|v|=1$, then $|v||u|=1$ and we can reduce the equation to:

$$ \theta = \arccos {({v \cdot u})} $$

This means we could also normalize the vectors (reduce the length to 1) and calculate the angle between those normalized vectors.

<- Up Next⌗

This is the first post in a series. In the next posts, we’ll be exploring other methods of calculating the angle with some special approaches, along with visualizations of what exactly we’re doing when we apply linear algebra concepts to this problem.

<- Read the next post