9.3  The Dot Product

Section 9.2 gave Rn two operations, adding points and scaling them, and both produce points. Nothing so far measures anything: there is no length, no distance and no right angle. This section introduces one more operation, which takes two points and returns a number, and proves the rules it obeys. Length, distance and angle are built from it in the rest of the chapter.

9.3.1. A product that is a number

Start with the plane. By Pythagoras, the point (x1,x2) lies at distance x12+x22 from the origin, and Chapter 3 gave this number a name: it is the modulus |x1+x2i| of Definition 3.2.1. Pythagoras is the motivation here, not a reason; nothing below rests on a picture.

Look at the expression under the root. It is a sum of products of coordinates. Each coordinate is multiplied by itself: x12+x22=x1x1+x2x2. Take this apart for two points: keep the first factor of each product from x and take the second from another point y. The result x1y1+x2y2 still makes sense, in Rn for every n, and it is a real number, not a point. The squared length is the special case y=x.

For instance, take x=(2,1) and y=(1,4). The mixed sum is 2⋅1+1⋅4=6. With y=x it is 2⋅2+1⋅1=5, the square of the distance 5 of (2,1) from the origin.

Multiply coordinate by coordinate, then add.

Definition 9.3.1 (Dot Product)

Let n≥1. The dot product of x=(x1,…,xn) and y=(y1,…,yn) in Rn is the real number ⟨x,y⟩:=∑k=1nxkyk=x1y1+⋯+xnyn.

In words: pair the coordinates place by place, multiply each pair in R, and add the n products. The two inputs must lie in the same Rn; for x∈R2 and y∈R3 the dot product is not defined. You may also see the notation x⋅y. This book always writes ⟨x,y⟩, with two slots and a comma, so that the product of two points is never confused with the scalar multiple cx of Section 9.2. The sum is the finite sum of Sets and Functions, Definition 3.3.1. The definition needs no further check, since the coordinates of x and y are determined by the points (Definition 9.1.1).

Examples.

In the plane. ⟨(1,2),(3,4)⟩=1⋅3+2⋅4=11, and ⟨(3,4),(3,4)⟩=9+16=25, the square of the modulus |3+4i|=5 of Chapter 3.

Signs. ⟨(1,2),(−2,1)⟩=−2+2=0, and ⟨(1,0),(−1,0)⟩=−1. The dot product can be zero or negative even when neither point is 0.

On the line, n=1. A sum with one term is that term (∑k=11ak=0+a1=a1), so for x,y∈R=R1 the dot product is the ordinary product: ⟨x,y⟩=xy. This is why the definition deserves the name product. On the line, the dot product of a point with itself is x2≥0; in Rn it is a sum of such squares, by law (c) below.

Change one sign. On R2, try B(x,y):=x1y1−x2y2. It is still symmetric, and it still respects sums and scalar multiples in each slot. So it satisfies laws (a) and (b) of the next proposition. Law (c) fails: B(e2,e2)=−1<0, and B((1,1),(1,1))=0 although (1,1)≠0. So B cannot be the square of a length. The plus signs of the definition are what make ⟨x,x⟩ a sum of squares.

A number, not a point. Two features are deliberate. First, the output is a number. A product returning a point, such as the coordinatewise product (x1y1,…,xnyn), would not measure anything, and Section 9.2 already set products of points aside: in this chapter (1,2)(3,4) has no meaning. Second, the dot product of a point with itself is ∑kxk2, the expression under Pythagoras’ root. That is exactly what Section 9.4 needs to define length.

9.3.2. The laws of the dot product

Every law below reduces, term by term, to a law of R. The sum rules of Lemma 1.1.3 then collect the terms. For the order facts we use the comparison rules for sums of Chapter 1 (Proposition 1.1.5).

Those rules are stated for sums indexed from 0, while ours run from 1 to n. Sets and Functions showed how to shift the index of a sum (Sets and Functions, Definition 3.3.1, with p=−1). Here the shift gives ∑k=1nak=∑k=0n−1ak+1, and n−1∈N because n≥1. So each rule holds for sums from 1 to n as well, and we apply it in that form without repeating the shift.

First a small case. Take x=(1,2) and y=(3,4) in R2. Then ⟨x,y⟩=3+8=11 and ⟨y,x⟩=3+8=11: the order does not matter. Doubling x doubles the result, since ⟨(2,4),(3,4)⟩=6+16=22=2⋅11. And ⟨x,x⟩=1+4=5 is a sum of squares, so it is not negative. The proposition says that these laws hold for all points of every Rn.

Proposition 9.3.1 (Laws of the Dot Product)

Let n≥1, let x,y,z∈Rn and c∈R, and let e1,…,en be the standard vectors (Definition 9.2.2).

  • (a) (Symmetry.) ⟨x,y⟩=⟨y,x⟩.
  • (b) (Linearity in each slot.) ⟨x+y,z⟩=⟨x,z⟩+⟨y,z⟩ and ⟨cx,z⟩=c⟨x,z⟩; and likewise in the second slot, ⟨z,x+y⟩=⟨z,x⟩+⟨z,y⟩ and ⟨z,cx⟩=c⟨z,x⟩.
  • (c) (Positivity.) ⟨x,x⟩=∑k=1nxk2≥0, and ⟨x,x⟩=0 if and only if x=0.
  • (d) (Standard vectors.) ⟨x,ek⟩=xk for every k, and ⟨ej,ek⟩=1 if j=k, ⟨ej,ek⟩=0 if j≠k.
  • (e) (Zero.) ⟨x,0⟩=⟨0,x⟩=0.

Idea. We work coordinatewise, as in Example (Method: work coordinatewise), but now inside a sum. Write out each side as a sum over k. Apply the law of R to the k-th term. Then let the sum rules collect the terms. Part (c) is the one law with content: a sum of squares in R is ≥0, and it is 0 only when every square is 0.

Proof. Throughout, ⟨x,y⟩=∑k=1nxkyk, and the coordinates of x+y and cx are xk+yk and cxk.

  • (a) Each term satisfies xkyk=ykxk, because multiplication in R is commutative, (F6). So the two sums have the same terms and are equal.

  • (b) For sums, the distributive law, (F9), together with commutativity gives (xk+yk)zk=xkzk+ykzk for each k. A sum of sums splits into two sums, so ⟨x+y,z⟩=∑k=1n(xkzk+ykzk)=∑k=1nxkzk+∑k=1nykzk. For scalars, (cxk)zk=c(xkzk) because multiplication is associative, (F5). A constant factor comes out of a finite sum, so ∑kc(xkzk)=c∑kxkzk. The second-slot rules follow from the first-slot rules and (a): ⟨z,x+y⟩=⟨x+y,z⟩=⟨x,z⟩+⟨y,z⟩=⟨z,x⟩+⟨z,y⟩, and in the same way ⟨z,cx⟩=c⟨z,x⟩.

  • (c) The zero vector is the scalar 0 times the zero vector: 0=0⋅0 (Proposition 9.2.1 (i)). So (b) gives ⟨0,x⟩=⟨0⋅0,x⟩=0⋅⟨0,x⟩=0, since a product with a factor 0 is 0 in R. And ⟨x,0⟩=⟨0,x⟩ by (a). We prove (e) before (c), which uses it.

  • (d) Here xkyk with y=x is xkxk=xk2, which gives the formula. Each term is ≥0. If xk≠0, then xk2>0, since the square of a non-zero number is positive. If xk=0, then xk2=0. A sum of non-negative terms is non-negative, the empty sum being 0. So ⟨x,x⟩≥0.

(⇐) If x=0, then ⟨x,x⟩=⟨0,0⟩=0 by (e).

(⇒) Suppose ⟨x,x⟩=0. This is a sum of non-negative terms equal to 0, so every term is 0 (Proposition 1.1.5 (e)): xkxk=0 for each k. Then xk=0, since only 0 has square 0. Every coordinate of x is 0, so x=0 by equality of tuples.

  • (d) Fix k. The j-th coordinate of ek is 1 if j=k and 0 otherwise (Definition 9.2.2). So the terms of ⟨x,ek⟩=∑j=1nxj(ek)j are xj⋅0=0 for j≠k and xk⋅1=xk for j=k. A sum with one non-zero term is that term (Lemma 1.1.3 (d)), so ⟨x,ek⟩=xk. Taking x=ej gives ⟨ej,ek⟩=(ej)k, which is 1 if j=k and 0 otherwise. This proves the proposition.

Uses: Definition 9.3.1; Definition 9.2.1 (coordinates of x+y, cx); Number Systems: From N to R, Definition 2.3.1 (F5), (F6), (F9); Lemma 1.1.3 (a), (b) (splitting sums, constant factors); Number Systems: From N to R, Proposition 2.3.2 (b) (a⋅0=0); Number Systems: From N to R, Proposition 2.4.3 (e), (f) (squares); Proposition 1.1.5 (b) with m=−1 (sums of terms ≥0); Definition 9.1.1 (equality of tuples).

Part (d) says that the dot product with ek reads off the k-th coordinate; we use it whenever a coordinate has to be recovered from dot products. Part (c) is what will make length meaningful in Section 9.4: ⟨x,x⟩ has a square root, and only the zero vector has length 0.

One consequence of (a) and (b) is used constantly from the next section on: a dot product of sums can be multiplied out like a product of numbers. For x,y∈Rn, ⟨x+y,x+y⟩=⟨x,x+y⟩+⟨y,x+y⟩=⟨x,x⟩+⟨x,y⟩+⟨y,x⟩+⟨y,y⟩=⟨x,x⟩+2⟨x,y⟩+⟨y,y⟩. The first line is linearity in the first slot, the second is linearity in the second slot, and the third is symmetry, ⟨y,x⟩=⟨x,y⟩, with a+a=2a in R. Replacing y by −y=(−1)y changes the middle term only. By (b), ⟨x,−y⟩=−⟨x,y⟩ and ⟨−y,−y⟩=(−1)2⟨y,y⟩=⟨y,y⟩. So ⟨x−y,x−y⟩=⟨x,x⟩−2⟨x,y⟩+⟨y,y⟩. These two expansions are the dot-product form of (a±b)2=a2±2ab+b2 (Number Systems: From N to R, Proposition 2.3.2 (d)), and the middle term 2⟨x,y⟩ is the one that matters.

9.3.3. Orthogonal vectors

Take the triangle with sides x, y and x+y. The expansion above says when the plane’s Pythagorean relation holds for it. The dot product of x+y with itself is ⟨x,x⟩+⟨y,y⟩ exactly when 2⟨x,y⟩=0. Since 2≠0 in R, that happens exactly when ⟨x,y⟩=0.

In the plane we see the same thing. The points (1,2) and (−2,1) look perpendicular, since the second is the first turned a quarter turn about the origin. Their dot product is −2+2=0. So “dot product zero” does the job of a right angle. It makes sense in every Rn, where there is no picture to consult.

Two vectors are orthogonal when their dot product is zero.

Definition 9.3.2 (Orthogonal Vectors)

Let n≥1. Vectors x,y∈Rn are orthogonal, written x⊥y, if ⟨x,y⟩=0.

In words: one number, the dot product, must vanish. The relation is symmetric, x⊥y if and only if y⊥x, by Proposition 9.3.1 (a), so “x and y are orthogonal” is unambiguous. “Perpendicular” means the same. This book says “orthogonal”, the word used in linear algebra.

Examples.

Standard vectors. ej⊥ek whenever j≠k, by Proposition 9.3.1 (d).

In R3. (1,1,0)⊥(1,−1,5), since 1−1+0=0. The third coordinate of the second vector is irrelevant because the first vector’s third coordinate is 0.

The zero vector. 0⊥x for every x, by Proposition 9.3.1 (e). This matters: the zero vector is orthogonal to everything, including itself, so “orthogonal” does not mean “pointing in different directions”. And 0 is the only vector orthogonal to itself: x⊥x means ⟨x,x⟩=0, which forces x=0 by Proposition 9.3.1 (c).

Moving one coordinate. Change (−2,1) to (−2,2). Then ⟨(1,2),(−2,2)⟩=−2+4=2≠0, so (1,2) and (−2,2) are not orthogonal. The one condition ⟨x,y⟩=0 fails: the dot product is 2.

One equation, linear in each vector. We could have defined a right angle by a picture, or by the Pythagorean relation itself. The dot product is better: it is one equation, linear in each vector, so orthogonality to a fixed a is the solution set of ⟨a,x⟩=0, an equation in the coordinates of x. Chapter 10 builds its hyperplanes from equations of exactly this form.

Orthogonality also gives a way to prove that two vectors are equal. A vector orthogonal to everything is zero, and two vectors with the same dot products are the same vector.

Proposition 9.3.2 (The Orthogonality Test)

Let n≥1 and x,y∈Rn.

  • (a) If ⟨x,z⟩=0 for every z∈Rn, then x=0.
  • (b) The following are equivalent: (i) x=y; (ii) ⟨x,z⟩=⟨y,z⟩ for every z∈Rn; (iii) ⟨x,ek⟩=⟨y,ek⟩ for every k∈{1,…,n}.

Proof.

  • (a) The hypothesis holds for every z, in particular for z=x. So ⟨x,x⟩=0, and x=0 by Proposition 9.3.1 (c).

  • (b)
    • (i) ⇒ (ii): equal vectors have equal dot products with every z. (ii) ⇒ (iii): take z=ek. (iii) ⇒ (i): by Proposition 9.3.1 (d), (iii) says xk=yk for every k. Tuples that agree in every place are equal (Definition 9.1.1).

The hypothesis of (a) is a “for every z” statement, and the proof uses it once, at the one vector that settles the question.

Example (Dot products and orthogonality computed)
  • (a) Compute ⟨(1,2,−1),(3,0,4)⟩. (b) Show that (1,1,1,1)⊥(1,−1,1,−1) in R4. (c) Find all x∈R2 with x⊥(1,1). (d) In R3, when is x⊥e3?

Solution.

  • (a) 1⋅3+2⋅0+(−1)⋅4=3+0−4=−1.

  • (b) 1−1+1−1=0, so the two vectors are orthogonal. Neither is 0, and no coordinate of either is 0: orthogonality comes from cancellation between terms, not from zero entries.

  • (c) x⊥(1,1) means x1+x2=0. We show that the solutions are exactly the vectors t(1,−1) with t∈R. (⊆) If x1+x2=0, then x2=−x1, so x=(x1,−x1)=x1(1,−1); take t=x1. (⊇) ⟨t(1,−1),(1,1)⟩=t(1−1)=0 by Proposition 9.3.1 (b). So the vectors orthogonal to (1,1) are the multiples of (1,−1), and only those.

  • (d) By Proposition 9.3.1 (d), ⟨x,e3⟩=x3. So x⊥e3 if and only if x3=0: the vectors orthogonal to e3 are those of the form (x1,x2,0).

In part (c) the vectors orthogonal to the non-zero vector (1,1) are the multiples of one vector. In Rn, the solutions of one equation ⟨a,x⟩=c with a≠0 are what Chapter 10 calls a hyperplane.

Warning. The dot product does not obey every law of multiplication in R. There is no cancellation: ⟨e1,e2⟩=⟨e1,2e2⟩=0, yet e2≠2e2. There is no associativity: ⟨x,y⟩z and ⟨y,z⟩x are both defined, but for x=y=e1 and z=e2 the first is 1⋅e2=e2 and the second is 0⋅e1=0. And ⟨x,y⟩=0 does not force a zero factor: e1 and e2 are both non-zero. What does hold is Proposition 9.3.2: cancellation is allowed when the equation holds for every z.

Quick check. Is (2,−1,3)⊥(1,5,1)? What is ⟨x,x⟩ for x=(1,−2,0,3)?

Solution. Yes: 2−5+3=0. And ⟨x,x⟩=1+4+0+9=14, a sum of squares, as Proposition 9.3.1 (c) says it must be.

9.3.4. Exercises

A. Check your understanding

Exercise (A1)
  • (a) Compute ⟨(1,−2,0,3),(4,1,7,−1)⟩.
  • (b) True or false for all x,y∈Rn, with a reason: ⟨2x,2y⟩=2⟨x,y⟩.
  • (c) True or false for all x∈Rn, with a reason: if ⟨x,x⟩=0, then x=0.
  • (d) True or false for all x,y∈Rn, with a reason: if ⟨x,y⟩=0, then x=0 or y=0.

Solution.

  • (a) 4−2+0−3=−1.

  • (b) False. By Proposition 9.3.1 (b) in each slot, ⟨2x,2y⟩=2⟨x,2y⟩=4⟨x,y⟩. For x=y=e1 the left side is 4 and 2⟨x,y⟩=2.

  • (c) True: this is the “only if” half of Proposition 9.3.1 (c).

  • (d) False when n≥2: ⟨e1,e2⟩=0, and neither vector is 0. (For n=1 the statement is true, since ⟨x,y⟩=xy and R has no zero divisors.)

B. Practice

Exercise (B1: Orthogonal to two vectors)

Find all x∈R3 that are orthogonal to both (1,1,0) and (0,1,1).

Solution. The conditions are x1+x2=0 and x2+x3=0. We show that the solutions are exactly the vectors t(1,−1,1), t∈R.

(⊆) If both hold, put t:=x1. Then x2=−x1=−t and x3=−x2=t, so x=(t,−t,t)=t(1,−1,1).

(⊇) By Proposition 9.3.1 (b), ⟨t(1,−1,1),(1,1,0)⟩=t(1−1+0)=0 and ⟨t(1,−1,1),(0,1,1)⟩=t(0−1+1)=0.

Hence the vectors orthogonal to both are the multiples of (1,−1,1).

Exercise (B2: The difference of squares)

Prove that ⟨x−y,x+y⟩=⟨x,x⟩−⟨y,y⟩ for all x,y∈Rn, naming the law used at each step. Deduce that ⟨x,x⟩=⟨y,y⟩ if and only if x−y⊥x+y.

Solution. Write x−y=x+(−1)y. In R, (−1)s=−s. So linearity in the first slot gives ⟨x−y,x+y⟩=⟨x,x+y⟩−⟨y,x+y⟩. In R, −(s+t)=(−1)(s+t)=−s−t by the distributive law. So linearity in the second slot turns this into ⟨x,x⟩+⟨x,y⟩−⟨y,x⟩−⟨y,y⟩. By symmetry, ⟨y,x⟩=⟨x,y⟩, so the two middle terms cancel in R, leaving ⟨x,x⟩−⟨y,y⟩.

For the deduction: x−y⊥x+y means ⟨x−y,x+y⟩=0 (Definition 9.3.2), which by the identity means ⟨x,x⟩−⟨y,y⟩=0, that is, ⟨x,x⟩=⟨y,y⟩. Each step is an equivalence, so both directions hold.

In the plane this says that the diagonals x+y and x−y of the parallelogram on x and y are orthogonal exactly when the two sides have equal “squared length”.

Uses: Definition 9.2.1 (x−y=x+(−1)y); Proposition 9.3.1 (a), (b) (symmetry, linearity); Number Systems: From N to R, Proposition 2.3.2 (c) ((−1)s=−(1⋅s)=−s); Number Systems: From N to R, Definition 2.3.1 (F9) (distributive law).

Exercise (B3: Solving one linear equation)

Let a∈Rn with a≠0, and let r∈R. Show that x:=r⟨a,a⟩a satisfies ⟨a,x⟩=r, and that it is the only solution of ⟨a,x⟩=r that is parallel to a.

Solution. The division is allowed. Since a≠0, ⟨a,a⟩≠0 by Proposition 9.3.1 (c), and ⟨a,a⟩≥0, so ⟨a,a⟩>0.

Existence. By linearity in the second slot, ⟨a,x⟩=r⟨a,a⟩⟨a,a⟩=r. Also x is parallel to a by Definition 9.2.3, since x=ca with c=r/⟨a,a⟩.

Uniqueness among vectors parallel to a. Let w be parallel to a with ⟨a,w⟩=r. Since a≠0, w=ta for some t∈R, as Section 9.2 showed for vectors parallel to a non-zero vector (Definition 9.2.3). Then r=⟨a,ta⟩=t⟨a,a⟩, so t=r/⟨a,a⟩ and w=x.

For n≥2 there are other solutions, not parallel to a. Choose k with ak≠0 and j≠k, and let v:=akej−ajek. Then ⟨a,v⟩=akaj−ajak=0 by Proposition 9.3.1 (b), (d), and v≠0 because its j-th coordinate is ak≠0. So ⟨a,x+v⟩=r+0=r. If x+v were parallel to a, it would equal x by uniqueness, giving v=0. For example, with a=(1,1) and r=2: x=(1,1), and (2,0) is another solution.

C. Going deeper

Exercise (C1: Find the error)

Here is an argument.

Claim. Let x∈Rn. If ⟨x,z⟩=0 for every z∈Rn, then x=0.

Proof. Take z=e1. Then 0=⟨x,e1⟩=x1. Hence x=0.

Is the claim true? Does the argument prove it? If not, name the line that fails and what is wrong with it, and repair the argument.

Solution. The conclusion is true: it is Proposition 9.3.2 (a).

The argument does not prove it. The first two lines are correct: z=e1 is an allowed instance of the hypothesis, and ⟨x,e1⟩=x1 by Proposition 9.3.1 (d). The failing line is “Hence x=0”. To show x=0 we must show xk=0 for every k, and the argument checks one coordinate and concludes for all of them. This is the error of checking some instances and concluding “for all”, (N1) in the list of Logic and Proof (Logic and Proof, Example (What is not allowed)).

A counterexample to the step (not to the claim, which is true): in R2, the vector x=e2=(0,1) has x1=0, yet x≠0. So “x1=0, hence x=0” leads from a true line to a false one.

Repair. Either take z=x: then ⟨x,x⟩=0, and x=0 by Proposition 9.3.1 (c). Or take z=ek for each k∈{1,…,n}, which gives xk=0 for every k, and conclude by equality of tuples.

For n=1 the step happens to be valid, since x1 is the only coordinate; the argument is wrong because it is written for every n.