13.3 Gram Matrices

Chapter 11 attached to every list of vectors the matrix of its pairwise inner products and read three facts off it: it is Hermitian, its quadratic form is a squared length, and it is invertible exactly when the list is independent (Definition 11.2.3, Theorem 11.2.5). One question was left standing. Given a square matrix, is it the Gram matrix of something? The answer is the shortest possible one: exactly when it is positive semidefinite. This section proves that, and then spends it twice — the determinant of a Gram matrix is a squared volume, and a ratio of two Gram determinants is a distance to a subspace.

Throughout, $F = \mathbb{R}$ or $F = \mathbb{C}$ and all inner product spaces are over $F$ . When the list must be named we write $\mathbf{G}(v_1, \dots, v_k)$ for its Gram matrix, whose entries are $\langle v_j, v_i \rangle$ as in Definition 11.2.3.

13.3.1. Which matrices are Gram matrices?

Half of the answer is already proved. By Theorem 11.2.5 (a) and (b), a Gram matrix $\mathbf{G}$ is Hermitian and satisfies

$\mathbf{x}^{*}\mathbf{G}\mathbf{x} = \Big\lVert \sum_{j} x_jv_j \Big\rVert^2 \ \ge\ 0$

for every $\mathbf{x} \in F^k$ . That is word for word the definition of positive semidefiniteness (Definition 13.1.1). So being positive semidefinite is necessary. The content of the next theorem is that nothing else is needed, and this is where the chapter’s second signature move earns its keep: a positive semidefinite matrix factors as $\mathbf{B}^{*}\mathbf{B}$ , and the columns of $\mathbf{B}$ are then the list we were looking for.

Theorem 13.3.1 (Gram Matrices Are Exactly the Positive Semidefinite Matrices)

Let $\mathbf{G} \in M_k(F)$ . The following are equivalent.

(a) $\mathbf{G}$ is the Gram matrix of some list of $k$ vectors in some inner product space over $F$ ;
(b) $\mathbf{G}$ is the Gram matrix of some list of $k$ vectors in $F^{r}$ with the standard inner product, where $r = \operatorname{rank} \mathbf{G}$ ;
(c) $\mathbf{G} \succeq \mathbf{O}$ .

Moreover the list is unique up to an isometry: if $(v_1, \dots, v_k)$ in $V$ and $(w_1, \dots, w_k)$ in $W$ have the same Gram matrix, then there is an isometry from $\operatorname{span}(v_1, \dots, v_k)$ onto $\operatorname{span}(w_1, \dots, w_k)$ carrying $v_j$ to $w_j$ for every $j$ .

Idea.

(a) $\Rightarrow$ (c) is the display above. For (c) $\Rightarrow$ (b), diagonalize: the spectral theorem writes $\mathbf{G} = \mathbf{U}\mathbf{D}\mathbf{U}^{*}$ with $\mathbf{D}$ diagonal and, by Theorem 13.1.4, with nonnegative diagonal. Then $\mathbf{D}^{1/2}\mathbf{U}^{*}$ is a square root of $\mathbf{G}$ in the $\mathbf{B}^{*}\mathbf{B}$ sense, and its rows beyond the $r$ -th are zero, so the columns already live in $F^{r}$ . For the uniqueness clause, the Gram matrix is the recipe for every inner product among the vectors, so the obvious map $\sum a_jv_j \mapsto \sum a_jw_j$ has no choice but to preserve them; the only thing needing care is that it is well defined, and part (b) of Theorem 11.2.5 says precisely that.

Proof.

(b) $\Rightarrow$ (a) is immediate, since $F^{r}$ with the standard inner product is an inner product space.
(c) $\Rightarrow$ (c). Let $\mathbf{G}$ be the Gram matrix of $(v_1, \dots, v_k)$ . By Theorem 11.2.5 (a), $\mathbf{G}^{*} = \mathbf{G}$ , and by (b) of the same theorem $\mathbf{x}^{*}\mathbf{G}\mathbf{x} = \lVert\sum_j x_jv_j\rVert^2 \ge 0$ for every $\mathbf{x} \in F^k$ . Hence $\mathbf{G} \succeq \mathbf{O}$ by Definition 13.1.1.
(d) $\Rightarrow$ (b). Since $\mathbf{G}$ is Hermitian, the spectral theorem for matrices gives a unitary $\mathbf{U}$ and a diagonal $\mathbf{D}$ with $\mathbf{G} = \mathbf{U}\mathbf{D}\mathbf{U}^{*}$ (Corollary 12.4.3 over $\mathbb{C}$ ; Corollary 12.5.2 over $\mathbb{R}$ , with $\mathbf{U}$ orthogonal and $\mathbf{D}$ real). The diagonal entries of $\mathbf{D}$ are the eigenvalues of $\mathbf{G}$ , so they are real and $\ge 0$ by Theorem 13.1.4, and we may prescribe their order: let $d_1, \dots, d_r > 0$ and $d_{r+1} = \dots = d_k = 0$ . Here $r$ is the number of nonzero eigenvalues, which is $\operatorname{rank} \mathbf{G}$ , because $\mathbf{G}$ and $\mathbf{D}$ have the same rank.

Put $\mathbf{B} = \mathbf{D}^{1/2}\mathbf{U}^{*} \in M_k(F)$ , where $\mathbf{D}^{1/2} = \operatorname{diag}(\sqrt{d_1}, \dots, \sqrt{d_k})$ . Then $\mathbf{B}^{*}\mathbf{B} = \mathbf{U}\mathbf{D}^{1/2}\mathbf{D}^{1/2}\mathbf{U}^{*} = \mathbf{U}\mathbf{D}\mathbf{U}^{*} = \mathbf{G} ,$ using $(\mathbf{D}^{1/2})^{*} = \mathbf{D}^{1/2}$ . Rows $r+1, \dots, k$ of $\mathbf{B}$ are zero, since row $p$ of $\mathbf{D}^{1/2}\mathbf{U}^{*}$ is $\sqrt{d_p}$ times row $p$ of $\mathbf{U}^{*}$ . Let $\mathbf{B}' \in M_{r \times k}(F)$ consist of the first $r$ rows of $\mathbf{B}$ ; deleting zero rows changes no entry of $\mathbf{B}^{*}\mathbf{B}$ , so $(\mathbf{B}')^{*}\mathbf{B}' = \mathbf{G}$ as well. Let $\mathbf{v}_1, \dots, \mathbf{v}_k \in F^{r}$ be the columns of $\mathbf{B}'$ . With the standard inner product $\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{y}^{*}\mathbf{x}$ on $F^{r}$ , $\langle \mathbf{v}_j, \mathbf{v}_i \rangle = \mathbf{v}_i^{*}\mathbf{v}_j = \big((\mathbf{B}')^{*}\mathbf{B}'\big)_{ij} = (\mathbf{G})_{ij} ,$ so $\mathbf{G}$ is the Gram matrix of $(\mathbf{v}_1, \dots, \mathbf{v}_k)$ .

The uniqueness clause. Let $(v_1, \dots, v_k)$ in $V$ and $(w_1, \dots, w_k)$ in $W$ both have Gram matrix $\mathbf{G}$ , and put $U = \operatorname{span}(v_1, \dots, v_k)$ and $U' = \operatorname{span}(w_1, \dots, w_k)$ . Define $T \colon U \to U'$ by $T\Big(\sum_{j} a_jv_j\Big) = \sum_{j} a_jw_j .$ This is well defined: if $\sum_j a_jv_j = \sum_j b_jv_j$ , let $\mathbf{x} \in F^k$ be the vector with entries $x_j = a_j - b_j$ ; then Theorem 11.2.5 (b), applied to each list in turn, gives $\Big\lVert \sum_{j} x_jw_j \Big\rVert^2 = \mathbf{x}^{*}\mathbf{G}\mathbf{x} = \Big\lVert \sum_{j} x_jv_j \Big\rVert^2 = 0 ,$ so $\sum_j a_jw_j = \sum_j b_jw_j$ . The same display with every $b_j = 0$ shows $\left\| Tu \right\| = \left\| u \right\|$ for every $u \in U$ , so $T$ is an isometry (Definition 11.7.1); it is linear and surjective by its definition on spanning vectors. This proves the theorem.

Three consequences are worth naming at once.

First, the corollary promised in Chapter 7 is now proved from the right principle. Corollary 7.7.3 said that $\det(\mathbf{A}\mathbf{A}^{\top}) \ge 0$ for real $\mathbf{A}$ , with equality exactly when the rows of $\mathbf{A}$ are dependent, and it proved this by writing the determinant as a sum of squares of minors. The structural reason is the theorem above: $\mathbf{A}\mathbf{A}^{\top}$ is a Gram matrix, hence positive semidefinite, hence has nonnegative eigenvalues, and its determinant is their product. Positivity of the determinant is not an accident of Cauchy–Binet; it is one shadow of a much stronger property, and the other shadows come free — every principal minor of a Gram matrix is $\ge 0$ too, since a principal submatrix of $\mathbf{G}(v_1, \dots, v_k)$ is the Gram matrix of a sublist.

Second, the smallest space that can hold a realization of $\mathbf{G}$ has dimension $\operatorname{rank} \mathbf{G}$ , and no smaller one will do: a list of $k$ vectors in a space of dimension $s$ has a Gram matrix of rank at most $s$ , because $\mathbf{G} = \mathbf{A}^{*}\mathbf{A}$ for the $s \times k$ coordinate matrix $\mathbf{A}$ of the list and $\operatorname{rank}(\mathbf{A}^{*}\mathbf{A}) = \operatorname{rank} \mathbf{A} \le s$ by Corollary 11.6.6 (b).

Third, the theorem is a free source of examples in both directions: any vectors at all produce a positive semidefinite table, and conversely positivity is a real constraint on a table of numbers — a stronger one than most readers expect.

Warning. Plausible-looking symmetric tables need not be Gram matrices. The condition is not “symmetric with nonnegative diagonal”, and it is not even “every $2 \times 2$ principal minor is $\ge 0$ ”. Take $\mathbf{G} = \begin{pmatrix} 5 & -3 & -3 \\ -3 & 5 & -3 \\ -3 & -3 & 5 \end{pmatrix} .$ Its three $2 \times 2$ principal minors are all $25 - 9 = 16 > 0$ , so any two of the three vectors would be a legitimate pair. But $\mathbf{G}(1,1,1)^{\top} = (-1, -1, -1)^{\top}$ , so $-1$ is an eigenvalue and $\mathbf{G} \not\succeq \mathbf{O}$ : no three vectors in any inner product space have this table. In words, three vectors of squared length $5$ cannot be pairwise at an angle with cosine $-3/5$ ; two of them can.

Quick check. Is the $k \times k$ matrix with every entry equal to $1$ a Gram matrix? If so, exhibit a list; if not, say which clause fails.

Solution. It is. Take $v_1 = \dots = v_k = v$ for any unit vector $v$ : then $\langle v_j, v_i \rangle = 1$ for all $i, j$ . Consistently, the matrix has rank $1$ , and Theorem 13.3.1 (b) predicts a realization in $F^1$ ; the list $(1, 1, \dots, 1)$ in $F$ is one.

13.3.2. The Gram determinant is a squared volume

Chapter 7 defined the volume of a parallelepiped spanned by $n$ vectors of $\mathbb{R}^n$ as $\lvert\det\rvert$ of the matrix of their coordinates (Definition 7.9.1). That definition needs a square matrix, so it says nothing about two vectors in $\mathbb{R}^3$ . The missing notion was already present in Chapter 7’s motivation, though: volume is base times height. Turning that into a definition costs one line, now that we have distances to subspaces.

Definition 13.3.1 (Volume of a list)

Let $(v_1, \dots, v_k)$ be a list in an inner product space $V$ over $F$ . Its $k$ -dimensional volume $\operatorname{vol}(v_1, \dots, v_k) \ge 0$ is defined recursively by $\operatorname{vol}(v_1) = \left\| v_1 \right\|, \qquad \operatorname{vol}(v_1, \dots, v_k) = \operatorname{vol}(v_1, \dots, v_{k-1}) \cdot d(v_k, U_{k-1}) ,$ where $U_{k-1} = \operatorname{span}(v_1, \dots, v_{k-1})$ and $d(\cdot, \cdot)$ is the distance of Definition 11.3.3.

In words: the volume of the last vector’s parallelepiped is the volume of the one below it, times the height of the last vector above the subspace the others span. For $k = 2$ this is base times height for a parallelogram, and it is $0$ exactly when $v_2$ lies on the line through $v_1$ .

$Diagram. Labels: elem v_1, elem v_2, d(elem v_2, U_1), U = Span (elem v_1,elem v_2), base times height: vol (elem v_1,elem v_2) = norm elem v_1\,d(elem v_2, U_1)$

The recursion is easy to state and awkward to compute with, because each step needs a projection. The Gram determinant computes all of it at once.

Lemma 13.3.2 (Bordering a Gram matrix)

Let $(v_1, \dots, v_k)$ be a list in an inner product space $V$ , let $U = \operatorname{span}(v_1, \dots, v_k)$ , and let $w \in V$ . Then $\det \mathbf{G}(v_1, \dots, v_k, w) = \det \mathbf{G}(v_1, \dots, v_k)\cdot d(w, U)^2 .$

Idea. Replace $w$ by $w - P_Uw$ . That is a column operation on the list, and a column operation on the list is a congruence $\mathbf{G} \mapsto \mathbf{E}^{*}\mathbf{G}\mathbf{E}$ on the Gram matrix, with $\mathbf{E}$ unitriangular, so the determinant does not move. After the replacement the last vector is orthogonal to all the others, and the Gram matrix has become block diagonal.

Proof. First record how the Gram matrix responds to a change of list. If $u_j = \sum_t e_{tj}v_t$ for a matrix $\mathbf{E} = (e_{tj})$ , then by sesquilinearity $\langle u_j, u_i \rangle = \sum_{t}\sum_{s} e_{tj}\overline{e_{si}}\langle v_t, v_s \rangle ,$ which is the $(i, j)$ -entry of $\mathbf{E}^{*}\mathbf{G}\mathbf{E}$ ; that is, $\mathbf{G}(u_1, \dots, u_m) = \mathbf{E}^{*}\mathbf{G}(v_1, \dots, v_k)\mathbf{E}$ .

Now write $w = p + h$ with $p = P_Uw \in U$ and $h = w - p \in U^{\perp}$ , which is possible by Theorem 11.3.2 (a). Since $p \in U$ , there are scalars $c_1, \dots, c_k$ with $p = \sum_t c_tv_t$ . Let $\mathbf{E} \in M_{k+1}(F)$ be the identity matrix with its last column replaced by $(-c_1, \dots, -c_k, 1)$ . Applying the rule above to the list $(v_1, \dots, v_k, w)$ and this $\mathbf{E}$ turns it into $(v_1, \dots, v_k, h)$ , so $\mathbf{G}(v_1, \dots, v_k, h) = \mathbf{E}^{*}\,\mathbf{G}(v_1, \dots, v_k, w)\,\mathbf{E} .$ The matrix $\mathbf{E}$ is triangular with ones on the diagonal, so $\det \mathbf{E} = 1$ by Theorem 7.4.6, and $\det(\mathbf{E}^{*}) = \overline{\det \mathbf{E}} = 1$ . By Theorem 7.5.4 the two Gram determinants are equal.

Finally, $h \in U^{\perp}$ gives $\langle h, v_i \rangle = \langle v_i, h \rangle = 0$ for every $i$ , so $\mathbf{G}(v_1, \dots, v_k, h) = \begin{pmatrix} \mathbf{G}(v_1, \dots, v_k) & \mathbf{0} \\ \mathbf{0} & \left\| h \right\|^2 \end{pmatrix},$ whose determinant is $\det \mathbf{G}(v_1, \dots, v_k)\cdot\left\| h \right\|^2$ by cofactor expansion along the last row. Since $\left\| h \right\| = \left\| w - P_Uw \right\| = d(w, U)$ by Definition 11.3.3, this proves the lemma.

The recursion names the vectors in a particular order, and the last one plays a special role. The value does not depend on that choice: Theorem 13.3.3 below identifies $\operatorname{vol}(v_1, \dots, v_k)^2$ with $\det\mathbf{G}$ , and permuting the list permutes the rows of $\mathbf{G}$ and the columns in the same way, which leaves the determinant unchanged (Theorem 7.5.1 (a) applied once to the rows and once to the columns, so the two sign changes cancel). So the volume is an invariant of the list as a set of vectors, and we are free to order it however a computation prefers.

Theorem 13.3.3 (The Gram Determinant Is the Squared Volume)

Let $(v_1, \dots, v_k)$ be a list in an inner product space over $F$ . Then $\det \mathbf{G}(v_1, \dots, v_k) = \operatorname{vol}(v_1, \dots, v_k)^2 .$ In particular, if $\mathbf{A} \in M_{m \times k}(F)$ has columns $\mathbf{v}_1, \dots, \mathbf{v}_k \in F^m$ , then $\operatorname{vol}(\mathbf{v}_1, \dots, \mathbf{v}_k) = \sqrt{\det(\mathbf{A}^{*}\mathbf{A})}$ .

Proof. Induction on $k$ . For $k = 1$ , $\mathbf{G}(v_1) = (\left\| v_1 \right\|^2)$ and $\operatorname{vol}(v_1) = \left\| v_1 \right\|$ .

Let $k \ge 2$ and assume the statement for $k - 1$ . With $U_{k-1} = \operatorname{span}(v_1, \dots, v_{k-1})$ , Lemma 13.3.2 and then the induction hypothesis give $\begin{aligned} \det \mathbf{G}(v_1, \dots, v_k) &= \det \mathbf{G}(v_1, \dots, v_{k-1})\cdot d(v_k, U_{k-1})^2 \\ &= \operatorname{vol}(v_1, \dots, v_{k-1})^2 \cdot d(v_k, U_{k-1})^2 , \end{aligned}$ and the right-hand side is $\operatorname{vol}(v_1, \dots, v_k)^2$ by Definition 13.3.1.

For the last claim, $\langle \mathbf{v}_j, \mathbf{v}_i \rangle = \mathbf{v}_i^{*}\mathbf{v}_j = (\mathbf{A}^{*}\mathbf{A})_{ij}$ in $F^m$ with the standard inner product, so $\mathbf{G}(\mathbf{v}_1, \dots, \mathbf{v}_k) = \mathbf{A}^{*}\mathbf{A}$ , and the volume is the nonnegative square root of its determinant.

Two checks that the definition is the right one. If $k = m$ and $F = \mathbb{R}$ , then $\mathbf{A}$ is square and $\det(\mathbf{A}^{\top}\mathbf{A}) = (\det\mathbf{A})^2$ by Theorem 7.5.4 and Theorem 7.4.5, so $\operatorname{vol}(\mathbf{v}_1, \dots, \mathbf{v}_m) = \lvert\det\mathbf{A}\rvert$ : Definition 13.3.1 agrees with Definition 7.9.1 whenever both apply. And for $k = 2$ in $\mathbb{R}^3$ , Exercise (C1: Area of a triangle in space) defined the area of a parallelogram in space to be $\sqrt{\det(\mathbf{A}^{\top}\mathbf{A})}$ and promised a justification from inner products; the theorem is that justification, since base times height is what an area should be.

The volume vanishes exactly when the list is dependent, which is Theorem 11.2.5 (d) read through the theorem: a flat parallelepiped has no volume.

13.3.3. Distance to a subspace, by determinants

Dividing in Lemma 13.3.2 gives a formula for a distance with no projection in sight.

Theorem 13.3.4 (Distance by Gram Determinants)

Let $U$ be a finite-dimensional subspace of an inner product space $V$ , let $(v_1, \dots, v_k)$ be a basis of $U$ , and let $w \in V$ . Then $d(w, U)^2 = \frac{\det \mathbf{G}(v_1, \dots, v_k, w)}{\det \mathbf{G}(v_1, \dots, v_k)} .$

Proof. The list $(v_1, \dots, v_k)$ is independent, so $\det \mathbf{G}(v_1, \dots, v_k) > 0$ by Theorem 11.2.5 (d) and the quotient is defined. Dividing the identity of Lemma 13.3.2 by it gives the formula.

No orthonormal basis, no Gram–Schmidt, and no projection: two determinants and a division. The price is a $(k+1) \times (k+1)$ determinant in the numerator, so for large $k$ the projection route of Chapter 11 is faster. The quotient is also a ratio of squared volumes, $\operatorname{vol}(v_1, \dots, v_k, w)^2/\operatorname{vol}(v_1, \dots, v_k)^2$ : volume over base area, which is the height.

Example (The same distance twice)

In $\mathbb{R}^3$ with the dot product, let $U = \{(x, y, z) : x + y + z = 0\}$ and $\mathbf{w} = (1, 2, 3)$ . Compute $d(\mathbf{w}, U)$ by Theorem 13.3.4, and compare with Example (Distance from a point to a plane).

Solution. The vectors $\mathbf{v}_1 = (1, -1, 0)$ and $\mathbf{v}_2 = (0, 1, -1)$ lie in $U$ and are independent, and $\dim U = 2$ , so they are a basis of $U$ . Their inner products are $\langle \mathbf{v}_1, \mathbf{v}_1 \rangle = 2$ , $\langle \mathbf{v}_2, \mathbf{v}_2 \rangle = 2$ , $\langle \mathbf{v}_1, \mathbf{v}_2 \rangle = -1$ , and with $\mathbf{w}$ : $\langle \mathbf{w}, \mathbf{w} \rangle = 14$ , $\langle \mathbf{w}, \mathbf{v}_1 \rangle = 1 - 2 = -1$ , $\langle \mathbf{w}, \mathbf{v}_2 \rangle = 2 - 3 = -1$ . Hence $\mathbf{G}(\mathbf{v}_1, \mathbf{v}_2) = \begin{pmatrix} 2 & -1 \\ -1 & 2 \end{pmatrix}, \qquad \mathbf{G}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{w}) = \begin{pmatrix} 2 & -1 & -1 \\ -1 & 2 & -1 \\ -1 & -1 & 14 \end{pmatrix} .$ The first determinant is $4 - 1 = 3$ . Expanding the second along the first row, $2(28 - 1) + 1(-14 - 1) - 1(1 + 2) = 54 - 15 - 3 = 36 .$ So $d(\mathbf{w}, U)^2 = 36/3 = 12$ and $d(\mathbf{w}, U) = 2\sqrt3$ . Example (Distance from a point to a plane) obtained $\left\| (2,2,2) \right\| = 2\sqrt3$ by projecting onto the normal line. The two routes never meet on the page, and they agree.

Remark. The numerator $36$ is the squared volume of the parallelepiped on $\mathbf{v}_1, \mathbf{v}_2, \mathbf{w}$ , and the denominator $3$ is the squared area of its base. The determinant of the coordinate matrix of $(\mathbf{v}_1, \mathbf{v}_2, \mathbf{w})$ is $6$ , and indeed $6^2 = 36$ : Theorem 13.3.3 in the square case.

13.3.4. Exercises

A. Check your understanding

Exercise (A1)

(a) State the criterion for a matrix $\mathbf{G} \in M_k(F)$ to be the Gram matrix of some list of vectors.
(b) Determine whether the following statement is correct, and justify your answer: every symmetric matrix with nonnegative entries is a Gram matrix.
(c) A list of $5$ vectors in $\mathbb{R}^3$ has a $5 \times 5$ Gram matrix. What are the possible values of its rank, and what is its determinant?
(d) Write down the formula for $d(w, U)$ in terms of Gram determinants, saying what list is used.
(e) Define the $k$ -dimensional volume of a list, and say what Theorem 13.3.3 asserts about it.

Solution.

(a) $\mathbf{G}$ is a Gram matrix if and only if $\mathbf{G} \succeq \mathbf{O}$ (Theorem 13.3.1). The list may then be taken in $F^{r}$ with $r = \operatorname{rank}\mathbf{G}$ .
(b) Incorrect. $\begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ has nonnegative entries and eigenvalues $3$ and $-1$ , so it is not positive semidefinite and by Theorem 13.3.1 it is no Gram matrix. (Concretely: two unit vectors cannot have inner product $2$ , by Cauchy–Schwarz, Theorem 11.1.5.)
(c) The rank is $\operatorname{rank} \mathbf{G} = \dim\operatorname{span}(v_1, \dots, v_5) \le 3$ , so it is $0, 1, 2$ or $3$ ; in particular $\mathbf{G}$ is singular and $\det \mathbf{G} = 0$ . (A list of $5$ vectors in a $3$ -dimensional space is dependent, so Theorem 11.2.5 (d) gives $\det \mathbf{G} = 0$ directly.)
(d) If $(v_1, \dots, v_k)$ is a basis of $U$ , then $d(w, U)^2 = \frac{\det\mathbf{G}(v_1, \dots, v_k, w)}{\det\mathbf{G}(v_1, \dots, v_k)}$ by Theorem 13.3.4.
(e) $\operatorname{vol}(v_1) = \left\| v_1 \right\|$ and $\operatorname{vol}(v_1, \dots, v_k) = \operatorname{vol}(v_1, \dots, v_{k-1})\,d(v_k, \operatorname{span}(v_1, \dots, v_{k-1}))$ (Definition 13.3.1). Theorem 13.3.3 says $\det\mathbf{G}(v_1, \dots, v_k) = \operatorname{vol}(v_1, \dots, v_k)^2$ .

B. Practice

Exercise (B1: Which are Gram matrices?)

Determine which of the following are Gram matrices of some list of vectors in some real inner product space. For those that are, exhibit a list; for those that are not, give a reason.

(i) $\begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$
(ii) $\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$
(iii) $\begin{pmatrix} 2 & -1 & -1 \\ -1 & 2 & -1 \\ -1 & -1 & 2 \end{pmatrix}$
(iv) $\begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 1 & 1 & 1 \end{pmatrix}$

Solution.

(i) Yes. It is symmetric with eigenvalues $1$ and $3$ , both $> 0$ , so it is positive definite and Theorem 13.3.1 applies. A list: $\mathbf{v}_1 = (1, 1, 0)$ , $\mathbf{v}_2 = (1, 0, 1)$ in $\mathbb{R}^3$ , whose Gram matrix is exactly this.
(ii) No. It is not symmetric, so it is not Hermitian and Definition 13.1.1 fails at once; a Gram matrix over $\mathbb{R}$ is symmetric by Theorem 11.2.5 (a).
(iii) Yes. Its rank is $2$ : the rows sum to zero, and the top-left $2 \times 2$ minor is $3 \ne 0$ . Its eigenvalues are $0$ (eigenvector $(1,1,1)$ ) and $3$ (twice, on the plane $x + y + z = 0$ ), all $\ge 0$ . A list in $\mathbb{R}^2$ , as Theorem 13.3.1 (b) promises with $r = 2$ : the three vectors $(\sqrt2, 0)$ , $(-\tfrac{1}{\sqrt2}, \sqrt{\tfrac32})$ , $(-\tfrac{1}{\sqrt2}, -\sqrt{\tfrac32})$ , three vectors of length $\sqrt2$ at $120^{\circ}$ to each other.
(iv) No. It is symmetric, but its leading principal minors are $1, 1, -1$ , and the determinant of a positive semidefinite matrix is $\ge 0$ (its eigenvalues are $\ge 0$ ). So it is not positive semidefinite, and by Theorem 13.3.1 it is not a Gram matrix. (Its eigenvalues are $1$ and $1 \pm \sqrt2$ , one of which is negative.)

Exercise (B2: A volume two ways)

In $\mathbb{R}^3$ , let $\mathbf{v}_1 = (1,1,0)$ , $\mathbf{v}_2 = (1,0,1)$ , $\mathbf{v}_3 = (0,1,1)$ . Compute $\operatorname{vol}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3)$ with Theorem 13.3.3, and check the answer against Definition 7.9.1.

Solution. The pairwise dot products give $\mathbf{G} = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{pmatrix},$ and expanding along the first row, $\det\mathbf{G} = 2(4-1) - 1(2-1) + 1(1-2) = 6 - 1 - 1 = 4$ . By Theorem 13.3.3, $\operatorname{vol}(\mathbf{v}_1,\mathbf{v}_2,\mathbf{v}_3) = \sqrt4 = 2$ .

Directly, the matrix $\mathbf{A}$ with these columns has $\det \mathbf{A} = 1(0-1) - 1(1-0) + 0 = -2$ , so Definition 7.9.1 gives $\lvert -2\rvert = 2$ . The two agree, as they must, since $\det(\mathbf{A}^{\top}\mathbf{A}) = (\det\mathbf{A})^2$ .

Exercise (B3: A distance in

\mathbb{R}^4

)

In $\mathbb{R}^4$ with the dot product, let $U = \operatorname{span}\big((1,1,0,0), (0,1,1,0)\big)$ and $\mathbf{w} = (1,0,0,1)$ . Find $d(\mathbf{w}, U)$ using Theorem 13.3.4.

Solution. Write $\mathbf{v}_1 = (1,1,0,0)$ and $\mathbf{v}_2 = (0,1,1,0)$ ; they are independent, hence a basis of $U$ . The inner products are $\langle \mathbf{v}_1, \mathbf{v}_1 \rangle = \langle \mathbf{v}_2, \mathbf{v}_2 \rangle = 2$ , $\langle \mathbf{v}_1, \mathbf{v}_2 \rangle = 1$ , $\langle \mathbf{w}, \mathbf{w} \rangle = 2$ , $\langle \mathbf{w}, \mathbf{v}_1 \rangle = 1$ , $\langle \mathbf{w}, \mathbf{v}_2 \rangle = 0$ . So $\mathbf{G}(\mathbf{v}_1, \mathbf{v}_2) = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}, \qquad \mathbf{G}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{w}) = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & 0 \\ 1 & 0 & 2 \end{pmatrix},$ with determinants $3$ and $2(4) - 1(2) + 1(-2) = 4$ . Hence $d(\mathbf{w}, U)^2 = 4/3$ and $d(\mathbf{w}, U) = 2/\sqrt3$ .

Check. The projection is $P_U\mathbf{w} = \tfrac13(2, 1, -1, 0)$ , so $\mathbf{w} - P_U\mathbf{w} = \tfrac13(1, -1, 1, 3)$ , of norm $\tfrac13\sqrt{12} = 2/\sqrt3$ .

C. Going deeper

Exercise (C1: Rank of a Gram matrix)

Let $(v_1, \dots, v_k)$ be a list in a finite-dimensional inner product space $V$ over $F$ , with Gram matrix $\mathbf{G}$ .

(a) Prove that $\operatorname{rank} \mathbf{G} = \dim\operatorname{span}(v_1, \dots, v_k)$ .
(b) Deduce Theorem 11.2.5 (c): $\mathbf{G}$ is invertible if and only if the list is independent.

Hint for (a): the coordinate matrix of the list in an orthonormal basis of the span.

Solution.

(a) Let $U = \operatorname{span}(v_1, \dots, v_k)$ , $r = \dim U$ , and let $(e_1, \dots, e_r)$ be an orthonormal basis of $U$ , which exists by Theorem 11.2.2. Let $\mathbf{A} \in M_{r \times k}(F)$ have as its $j$ -th column the coordinate vector of $v_j$ . As computed in the proof of Theorem 11.2.5 (d), $\mathbf{G} = \mathbf{A}^{*}\mathbf{A}$ . By Corollary 11.6.6 (b), $\operatorname{rank}\mathbf{G} = \operatorname{rank}(\mathbf{A}^{*}\mathbf{A}) = \operatorname{rank}\mathbf{A}$ . Finally $\operatorname{rank}\mathbf{A} = r$ : the columns of $\mathbf{A}$ span $F^{r}$ , because the coordinate map $U \to F^{r}$ is an isomorphism carrying the spanning list $(v_1, \dots, v_k)$ of $U$ to the columns of $\mathbf{A}$ .
(b) $\mathbf{G}$ is a $k \times k$ matrix, so it is invertible if and only if $\operatorname{rank}\mathbf{G} = k$ (Theorem 7.5.3 (d)). By (a) that says $\dim\operatorname{span}(v_1, \dots, v_k) = k$ , which holds exactly when the list is independent.

Exercise (C2: Why the conjugate is not optional)

Over $\mathbb{C}$ , one might try to build a table of inner products with the bilinear expression $\mathbf{v}_i^{\top}\mathbf{v}_j$ , without any conjugate. Let $\mathbf{v} = (1, i) \in \mathbb{C}^2$ .

(a) Compute $\mathbf{v}^{\top}\mathbf{v}$ and $\mathbf{v}^{*}\mathbf{v}$ , and explain why only one of them is a legitimate squared length.
(b) The list $(\mathbf{v})$ is independent, yet the $1 \times 1$ “bilinear Gram matrix” $(\mathbf{v}^{\top}\mathbf{v})$ is singular. Which hypothesis of Theorem 13.3.1 is being violated, and which statement of Chapter 7 does this illustrate?

Solution.

(a) $\mathbf{v}^{\top}\mathbf{v} = 1^2 + i^2 = 0$ , while $\mathbf{v}^{*}\mathbf{v} = \lvert 1\rvert^2 + \lvert i\rvert^2 = 2$ . A squared length must be $\ge 0$ and must vanish only for the zero vector (Definition 11.1.1), and $\mathbf{v} \ne \mathbf{0}$ ; so $\mathbf{v}^{\top}\mathbf{v}$ cannot be one, and $\mathbf{v}^{*}\mathbf{v} = \left\| \mathbf{v} \right\|^2$ is the correct value.
(b) Theorem 13.3.1 is a statement about the Gram matrix of Definition 11.2.3, whose entries are $\langle \mathbf{v}_j, \mathbf{v}_i \rangle = \mathbf{v}_i^{*}\mathbf{v}_j$ . The matrix $(\mathbf{v}^{\top}\mathbf{v}) = \mathbf{O}_1$ is not that matrix, so the theorem simply does not apply; the genuine Gram matrix here is $(2)$ , which is positive definite, as it must be for an independent list. This is exactly the failure recorded after Corollary 7.7.3: over $\mathbb{C}$ , $\mathbf{A} = \begin{pmatrix} 1 & i \end{pmatrix}$ has rank $1$ while $\mathbf{A}\mathbf{A}^{\top} = \mathbf{O}_1$ , so the real equality case of that corollary has no complex analogue — unless the transpose is replaced by the conjugate transpose.

Exercise (C3: Height and volume)

Let $(v_1, \dots, v_k)$ be an independent list in an inner product space over $F$ . For each $j$ , write $U_j$ for the span of $v_1, \dots, v_j$ .

(a) Prove that $\operatorname{vol}(v_1, \dots, v_k) \le \left\| v_1 \right\|\left\| v_2 \right\|\cdots\left\| v_k \right\|$ , with equality if and only if the list is orthogonal.
(b) Deduce that $\det\mathbf{G}(v_1, \dots, v_k) \le \left\| v_1 \right\|^2\cdots\left\| v_k \right\|^2$ .

Hint for (a): compare each height with the corresponding length.

Solution.

(a) Fix $j \ge 2$ and write $v_j = P_{U_{j-1}}v_j + h_j$ with $h_j \in U_{j-1}^{\perp}$ , as in Theorem 11.3.2 (a). The two summands are orthogonal, so Theorem 11.1.3 gives $\left\| v_j \right\|^2 = \left\| P_{U_{j-1}}v_j \right\|^2 + \left\| h_j \right\|^2 \ \ge\ \left\| h_j \right\|^2 = d(v_j, U_{j-1})^2 ,$ with equality if and only if $P_{U_{j-1}}v_j = 0$ . Multiplying the $k$ inequalities $d(v_j, U_{j-1}) \le \left\| v_j \right\|$ (with $d(v_1, U_0)$ read as $\left\| v_1 \right\|$ ) and using Definition 13.3.1 repeatedly gives $\operatorname{vol}(v_1, \dots, v_k) \le \left\| v_1 \right\|\cdots\left\| v_k \right\| .$ All factors are $> 0$ , since the list is independent, so equality holds if and only if it holds in every factor, that is $P_{U_{j-1}}v_j = 0$ for every $j \ge 2$ . That says each $v_j$ is orthogonal to $U_{j-1}$ , hence to every earlier $v_i$ ; and conversely an orthogonal list has $P_{U_{j-1}}v_j = 0$ by Theorem 11.3.3 (a).
(b) Square (a) and apply Theorem 13.3.3.