20.2 Other Inclusion Regions

The Gershgorin discs of §01 come from one row of $\mathbf{A}\mathbf{x} = \lambda\mathbf{x}$ , read at the largest coordinate of $\mathbf{x}$ . That leaves two kinds of information unused. The first is the second largest coordinate, which the triangle inequality simply replaced by the largest; using it gives a region of ovals that is never worse than the discs. The second is the geometry of the inner product, which the discs ignore entirely: the number $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ over unit vectors $\mathbf{x}$ fills a region of the plane, the numerical range, that contains every eigenvalue. This section builds both regions. Its main theorem is that the numerical range is always convex. It ends with a rectangle for the spectrum whose sides are eigenvalues of Hermitian matrices, and with a warning about what these regions cannot do.

Throughout, matrices are complex and $n \ge 1$ , and $\left\| \cdot \right\|$ on $\mathbb{C}^n$ is the Euclidean norm. As in §01, $r_i(\mathbf{A}) = \sum_{j \ne i}\lvert a_{ij}\rvert$ is the $i$ -th Gershgorin radius and $D_i(\mathbf{A})$ the $i$ -th disc (Definition 20.1.1). The plane $\mathbb{C}$ is treated as a real vector space when convexity is discussed, as Chapter 18 §01 does.

20.2.1. Using two coordinates: Brauer’s ovals

In the proof of Theorem 20.1.1 every $\lvert x_j\rvert$ with $j \ne k$ was bounded by the largest one, $\lvert x_k\rvert$ . That is wasteful when the second largest coordinate is much smaller. So keep two indices: $p$ , where $\lvert x_p\rvert$ is largest, and $q \ne p$ , where it is largest among the rest. Row $p$ bounds $\lvert\lambda - a_{pp}\rvert\lvert x_p\rvert$ by $r_p\lvert x_q\rvert$ , since every other coordinate is at most $\lvert x_q\rvert$ . Row $q$ bounds $\lvert\lambda - a_{qq}\rvert\lvert x_q\rvert$ by $r_q\lvert x_p\rvert$ . Multiplying the two bounds cancels the unknown coordinates. The result is a region defined by a product of distances.

Theorem 20.2.1 (Brauer’s Ovals of Cassini)

Let $n \ge 2$ and $\mathbf{A} \in M_n(\mathbb{C})$ . For $i \ne j$ put $O_{ij}(\mathbf{A}) \mathrel{:=} \bigl\{\, z \in \mathbb{C} : \lvert z - a_{ii}\rvert\,\lvert z - a_{jj}\rvert \le r_i(\mathbf{A})\,r_j(\mathbf{A}) \,\bigr\} .$

(a) Every eigenvalue of $\mathbf{A}$ lies in $\bigcup_{i \ne j} O_{ij}(\mathbf{A})$ .
(b) $O_{ij}(\mathbf{A}) \subseteq D_i(\mathbf{A}) \cup D_j(\mathbf{A})$ for every $i \ne j$ . Hence $\bigcup_{i \ne j}O_{ij}(\mathbf{A}) \subseteq \bigcup_i D_i(\mathbf{A})$ .

Idea. For (a), take the two largest coordinates of an eigenvector as above and multiply the two row estimates. One case needs separate care: if the second largest coordinate is $0$ , there is nothing to divide by, but then $\mathbf{x}$ is a multiple of $\mathbf{e}_p$ and $\lambda = a_{pp}$ , which lies in every oval through $a_{pp}$ . For (b), a point outside both discs is farther than $r_i$ from $a_{ii}$ and farther than $r_j$ from $a_{jj}$ , so the product of its distances exceeds $r_ir_j$ .

Proof.

(a) Let $\mathbf{A}\mathbf{x} = \lambda\mathbf{x}$ with $\mathbf{x} \ne \mathbf{0}$ . Choose $p$ with $\lvert x_p\rvert = \max_j\lvert x_j\rvert$ , which is positive, and then $q \ne p$ with $\lvert x_q\rvert = \max_{j \ne p}\lvert x_j\rvert$ ; this is possible because $n \ge 2$ .
Case 1: $x_q = 0$ . Then $x_j = 0$ for every $j \ne p$ , and row $p$ of $\mathbf{A}\mathbf{x} = \lambda\mathbf{x}$ reads $a_{pp}x_p = \lambda x_p$ . As $x_p \ne 0$ , $\lambda = a_{pp}$ . The left side of the inequality defining $O_{pq}(\mathbf{A})$ is then $0$ , and the right side is $\ge 0$ , so $\lambda \in O_{pq}(\mathbf{A})$ .

Case 2: $x_q \ne 0$ . Row $p$ gives $(\lambda - a_{pp})x_p = \sum_{j \ne p}a_{pj}x_j$ . Every $j \ne p$ has $\lvert x_j\rvert \le \lvert x_q\rvert$ , so by the triangle inequality (Theorem 0.6.4) $\lvert\lambda - a_{pp}\rvert\,\lvert x_p\rvert \le r_p(\mathbf{A})\,\lvert x_q\rvert .$ Row $q$ gives $(\lambda - a_{qq})x_q = \sum_{j \ne q}a_{qj}x_j$ . Every $j$ has $\lvert x_j\rvert \le \lvert x_p\rvert$ , so $\lvert\lambda - a_{qq}\rvert\,\lvert x_q\rvert \le r_q(\mathbf{A})\,\lvert x_p\rvert .$ Both sides of both inequalities are nonnegative, so they may be multiplied: $\lvert\lambda - a_{pp}\rvert\,\lvert\lambda - a_{qq}\rvert\,\lvert x_p\rvert\,\lvert x_q\rvert \le r_p(\mathbf{A})\,r_q(\mathbf{A})\,\lvert x_p\rvert\,\lvert x_q\rvert .$ Dividing by $\lvert x_p\rvert\lvert x_q\rvert > 0$ gives $\lambda \in O_{pq}(\mathbf{A})$ .
(b) Let $z \notin D_i(\mathbf{A}) \cup D_j(\mathbf{A})$ , so $a \mathrel{:=} \lvert z - a_{ii}\rvert > r_i(\mathbf{A}) \ge 0$ and $c \mathrel{:=} \lvert z - a_{jj}\rvert > r_j(\mathbf{A}) \ge 0$ . Since $c > 0$ and $r_i(\mathbf{A}) \ge 0$ , $ac > r_i(\mathbf{A})\,c \ge r_i(\mathbf{A})\,r_j(\mathbf{A}) ,$ so $z \notin O_{ij}(\mathbf{A})$ . Taking complements, $O_{ij}(\mathbf{A}) \subseteq D_i(\mathbf{A}) \cup D_j(\mathbf{A})$ , and taking the union over all pairs gives the last claim. This proves the theorem.

The sets $O_{ij}(\mathbf{A})$ are the ovals of Cassini: the points whose distances to two fixed foci have product at most a constant. Since $O_{ij}(\mathbf{A}) = O_{ji}(\mathbf{A})$ , there are $n(n-1)/2$ of them to draw, against $n$ discs. The hypothesis $n \ge 2$ is there because an oval needs two distinct indices; for $n = 1$ there are none, and the eigenvalue $a_{11}$ must be found some other way. For $n = 2$ the single oval is as good as a region can be. The eigenvalues satisfy $(\lambda - a_{11})(\lambda - a_{22}) = a_{12}a_{21}$ , so taking moduli, $\lvert\lambda - a_{11}\rvert\lvert\lambda - a_{22}\rvert = r_1r_2$ : both eigenvalues lie on the boundary curve of $O_{12}$ .

Example (Ovals where discs fail)

Let $\mathbf{A} = \begin{pmatrix} 4 & 1 & 0 \\ 0 & 4 & 1 \\ 1 & 1 & 1 \end{pmatrix}$ . Show that the Gershgorin discs cannot prove $\mathbf{A}$ invertible, and that the ovals can.

Solution. The radii are $r_1 = 1$ , $r_2 = 1$ , $r_3 = 2$ , and $D_3(\mathbf{A}) = \{\lvert z - 1\rvert \le 2\}$ contains $0$ . So Theorem 20.1.1 (a) allows the eigenvalue $0$ . The column radii are $1, 2, 1$ , and the second column disc $\lvert z - 4\rvert \le 2$ misses $0$ , but the third, $\lvert z - 1\rvert \le 1$ , contains it. So the discs cannot decide.

Now test $z = 0$ against the three ovals: $\begin{aligned} O_{12}&: \ \lvert 0 - 4\rvert\,\lvert 0 - 4\rvert = 16 > 1 = r_1r_2 ,\\ O_{13}&: \ \lvert 0 - 4\rvert\,\lvert 0 - 1\rvert = 4 > 2 = r_1r_3 ,\\ O_{23}&: \ \lvert 0 - 4\rvert\,\lvert 0 - 1\rvert = 4 > 2 = r_2r_3 . \end{aligned}$ So $0$ lies in no oval, it is not an eigenvalue by Theorem 20.2.1 (a), and $\mathbf{A}$ is invertible by Theorem 9.2.1. Indeed, expanding along the first row, $\det\mathbf{A} = 4(4 - 1) - 1(0 - 1) = 13$ . The ovals also exclude the point $-1$ , which lies on the boundary of $D_3(\mathbf{A})$ : there $\lvert -1 - 4\rvert\,\lvert -1 - 4\rvert = 25 > 1$ rules out $O_{12}$ , and $\lvert -1 - 4\rvert\,\lvert -1 - 1\rvert = 10 > 2$ rules out $O_{13}$ and $O_{23}$ .

The same test gives a criterion in the spirit of Corollary 20.1.2: if $n \ge 2$ , as in Theorem 20.2.1, and $\lvert a_{ii}\rvert\,\lvert a_{jj}\rvert > r_i(\mathbf{A})\,r_j(\mathbf{A})$ for every pair $i \ne j$ , then $0$ lies in no oval and $\mathbf{A}$ is invertible. One row may fail to dominate, as row $3$ did above, provided every other row dominates enough to compensate.

Quick check. Why does the proof of Theorem 20.2.1 (a) choose $q$ to maximize $\lvert x_q\rvert$ among the indices other than $p$ , rather than taking any $q \ne p$ ?

Solution. The estimate from row $p$ bounds every $\lvert x_j\rvert$ , $j \ne p$ , by $\lvert x_q\rvert$ . That is true only if $\lvert x_q\rvert$ is the largest of them. With an arbitrary $q$ the row- $p$ estimate could fail, and the product of the two estimates would no longer cancel down to $r_pr_q$ .

20.2.2. The numerical range

Now a different source of information. For a Hermitian $\mathbf{A}$ , Chapter 17 §01 studied the Rayleigh quotient, and Proposition 17.1.1 showed that its values $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ on unit vectors fill exactly the interval $[\lambda_n(\mathbf{A}), \lambda_1(\mathbf{A})]$ , which contains every eigenvalue. The number $\mathbf{x}^{*}\mathbf{A}\mathbf{x} = \langle \mathbf{A}\mathbf{x}, \mathbf{x} \rangle$ makes sense for every square matrix. Without the Hermitian hypothesis it is complex, and its values fill a region of the plane rather than an interval.

The numerical range is the set of all values $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ as $\mathbf{x}$ runs over the unit vectors.

Definition 20.2.1 (Numerical Range)

Let $\mathbf{A} \in M_n(\mathbb{C})$ . The numerical range of $\mathbf{A}$ is the subset of $\mathbb{C}$ $W(\mathbf{A}) \mathrel{:=} \{\, \mathbf{x}^{*}\mathbf{A}\mathbf{x} : \mathbf{x} \in \mathbb{C}^n,\ \left\| \mathbf{x} \right\| = 1 \,\} .$

In words: feed $\mathbf{A}$ a vector of length exactly $1$ , take the inner product of the output with the input, and collect all the complex numbers that arise. The vectors range over complex unit vectors, even when $\mathbf{A}$ is real, and the conjugate in $\mathbf{x}^{*}$ is part of the definition. $W(\mathbf{A})$ is always nonempty, since $\mathbf{e}_1$ is a unit vector, and $\mathbf{e}_i^{*}\mathbf{A}\mathbf{e}_i = a_{ii}$ , so every diagonal entry lies in $W(\mathbf{A})$ . The notation always carries its argument: $W(\mathbf{A})$ , never $W$ alone, since $W$ also names subspaces.

Examples, simplest first.

$n = 1$ , and scalar matrices. For $\mathbf{A} = (a)$ the unit vectors are the scalars $x$ with $\lvert x\rvert = 1$ , and $\overline{x}ax = a\lvert x\rvert^2 = a$ . So $W((a)) = \{a\}$ . Likewise $W(c\mathbf{I}) = \{c\}$ , since $\mathbf{x}^{*}(c\mathbf{I})\mathbf{x} = c\left\| \mathbf{x} \right\|^2 = c$ .
Diagonal matrices. For $\mathbf{A} = \operatorname{diag}(d_1, \dots, d_n)$ , $\mathbf{x}^{*}\mathbf{A}\mathbf{x} = \sum_i d_i\lvert x_i\rvert^2$ . The weights $t_i = \lvert x_i\rvert^2$ are nonnegative with sum $\left\| \mathbf{x} \right\|^2 = 1$ , and every such list of weights arises, from $x_i = \sqrt{t_i}$ . So $W(\mathbf{A})$ is the set of all weighted averages of $d_1, \dots, d_n$ , which is $\operatorname{conv}\{d_1, \dots, d_n\}$ by Theorem 18.1.3. For $\operatorname{diag}(1, i, -1)$ it is the filled triangle with these three vertices.
Hermitian matrices. By Proposition 17.1.1, $W(\mathbf{A}) = [\lambda_n(\mathbf{A}), \lambda_1(\mathbf{A})]$ , a segment of the real axis.
A Jordan block. The next example computes $W(\mathbf{J}_2(0))$ , and it is a whole disc, although the only eigenvalue is $0$ .

Example (The numerical range of a Jordan block)

Show that $W(\mathbf{J}_2(0)) = \{ z \in \mathbb{C} : \lvert z\rvert \le \tfrac12 \}$ .

Solution. Write $\mathbf{x} = (x_1, x_2)$ with $\lvert x_1\rvert^2 + \lvert x_2\rvert^2 = 1$ . Then $\mathbf{J}_2(0)\mathbf{x} = (x_2, 0)$ , so $\mathbf{x}^{*}\mathbf{J}_2(0)\mathbf{x} = \overline{x_1}\,x_2$ .

$(\subseteq)$ Since $0 \le (\lvert x_1\rvert - \lvert x_2\rvert)^2 = 1 - 2\lvert x_1\rvert\lvert x_2\rvert$ , we get $\lvert\overline{x_1}x_2\rvert = \lvert x_1\rvert\lvert x_2\rvert \le \tfrac12$ .

$(\supseteq)$ Let $z = \rho\,\omega$ with $0 \le \rho \le \tfrac12$ and $\lvert\omega\rvert = 1$ . Since $1 - 4\rho^2 \ge 0$ , we may put $a = \sqrt{\tfrac{1 + \sqrt{1 - 4\rho^2}}{2}}, \qquad b = \sqrt{\tfrac{1 - \sqrt{1 - 4\rho^2}}{2}} ,$ both real and nonnegative. Then $a^2 + b^2 = 1$ and $a^2b^2 = \tfrac14\bigl(1 - (1 - 4\rho^2)\bigr) = \rho^2$ , so $ab = \rho$ . The unit vector $\mathbf{x} = (a, b\,\omega)$ gives $\mathbf{x}^{*}\mathbf{J}_2(0)\mathbf{x} = a\,b\,\omega = z$ . Hence $W(\mathbf{J}_2(0))$ is the closed disc of radius $\tfrac12$ about $0$ .

Nonexample by minimal change. Drop the conjugate, and collect the numbers $\mathbf{x}^{\top}\mathbf{A}\mathbf{x}$ for unit $\mathbf{x}$ instead. For $\mathbf{A} = \mathbf{I}_2$ and the unit vector $\mathbf{x} = \tfrac{1}{\sqrt2}(1, i)$ , $\mathbf{x}^{\top}\mathbf{I}_2\mathbf{x} = \tfrac12(1 + i^2) = 0$ , while $\mathbf{x}^{*}\mathbf{I}_2\mathbf{x} = 1$ . The modified set for $\mathbf{I}_2$ contains $0$ , which is not an eigenvalue, while $W(\mathbf{I}_2) = \{1\}$ is exactly the spectrum. What breaks is that $\mathbf{x}^{\top}\mathbf{x}$ is not $\left\| \mathbf{x} \right\|^2$ over $\mathbb{C}$ . The conjugate is what makes $\mathbf{x}^{*}(c\mathbf{I})\mathbf{x} = c$ on the unit sphere, and every property below leans on that.

Why this definition. The unit length plays the role that division by $\left\| \mathbf{x} \right\|^2$ played for the Rayleigh quotient. Without it the values $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ over all $\mathbf{x}$ form a set closed under multiplication by positive reals, which is all of $\mathbb{C}$ or a cone, and records nothing about the size of $\mathbf{A}$ . With it, the set is bounded, and the eigenvalues are in it.

Proposition 20.2.2 (Basic Properties of the Numerical Range)

Let $\mathbf{A} \in M_n(\mathbb{C})$ .

(a) $\operatorname{spec}(\mathbf{A}) \subseteq W(\mathbf{A})$ .
(b) $W(\alpha\mathbf{A} + \beta\mathbf{I}) = \alpha W(\mathbf{A}) + \beta \mathrel{:=} \{\alpha w + \beta : w \in W(\mathbf{A})\}$ for all $\alpha, \beta \in \mathbb{C}$ .
(c) $W(\mathbf{U}^{*}\mathbf{A}\mathbf{U}) = W(\mathbf{A})$ for every unitary $\mathbf{U} \in M_n(\mathbb{C})$ .
(d) $W(\mathbf{A})$ is compact: every sequence in $W(\mathbf{A})$ has a subsequence converging to a point of $W(\mathbf{A})$ . In particular $W(\mathbf{A})$ is closed.
(e) $W(\mathbf{A}) \subseteq \{ z : \lvert z\rvert \le \left\| \mathbf{A} \right\|_2 \}$ .

Idea. Four of the five parts are one line of algebra with a unit vector: evaluate at a unit eigenvector for (a), push the scalars or the unitary through $\mathbf{x}^{*}(\cdot)\mathbf{x}$ for (b) and (c), and apply Cauchy–Schwarz for (e). Only (d) needs analysis, and there the point is that $W(\mathbf{A})$ is the image of the unit sphere under a continuous map, so the compactness of the sphere passes to it.

Proof.

(a) Let $\lambda \in \operatorname{spec}(\mathbf{A})$ , with eigenvector $\mathbf{v}$ . Then $\mathbf{x} = \mathbf{v}/\left\| \mathbf{v} \right\|$ is a unit eigenvector, and $\mathbf{x}^{*}\mathbf{A}\mathbf{x} = \lambda\,\mathbf{x}^{*}\mathbf{x} = \lambda$ .
(b) For a unit $\mathbf{x}$ , $\mathbf{x}^{*}(\alpha\mathbf{A} + \beta\mathbf{I})\mathbf{x} = \alpha\,\mathbf{x}^{*}\mathbf{A}\mathbf{x} + \beta\,\mathbf{x}^{*}\mathbf{x} = \alpha\,\mathbf{x}^{*}\mathbf{A}\mathbf{x} + \beta$ . As $\mathbf{x}$ runs over the unit vectors, the left side runs over $W(\alpha\mathbf{A} + \beta\mathbf{I})$ and the right side over $\alpha W(\mathbf{A}) + \beta$ .
(c) For a unit $\mathbf{x}$ , $\mathbf{x}^{*}\mathbf{U}^{*}\mathbf{A}\mathbf{U}\mathbf{x} = (\mathbf{U}\mathbf{x})^{*}\mathbf{A}(\mathbf{U}\mathbf{x})$ , and $\mathbf{U}\mathbf{x}$ is a unit vector because $\mathbf{U}$ is an isometry (Theorem 11.7.3). So $W(\mathbf{U}^{*}\mathbf{A}\mathbf{U}) \subseteq W(\mathbf{A})$ . Conversely, every unit $\mathbf{y}$ equals $\mathbf{U}\mathbf{x}$ for the unit vector $\mathbf{x} = \mathbf{U}^{*}\mathbf{y}$ , so every value $\mathbf{y}^{*}\mathbf{A}\mathbf{y}$ arises, and the two sets are equal.
(d) First, $f(\mathbf{x}) = \mathbf{x}^{*}\mathbf{A}\mathbf{x}$ is continuous on the unit sphere. For unit vectors $\mathbf{x}, \mathbf{y}$ , $\begin{aligned} \lvert f(\mathbf{x}) - f(\mathbf{y})\rvert &= \lvert \mathbf{x}^{*}\mathbf{A}(\mathbf{x} - \mathbf{y}) + (\mathbf{x} - \mathbf{y})^{*}\mathbf{A}\mathbf{y} \rvert \\ &\le 2\,\left\| \mathbf{A} \right\|_2\,\left\| \mathbf{x} - \mathbf{y} \right\| , \end{aligned}$ by the triangle inequality (Theorem 0.6.4), the Cauchy–Schwarz inequality (Theorem 11.1.5), and Theorem 16.3.2 (a). This is a bound of the kind fixed as the book’s continuity test in Chapter 16’s introduction. Now let $(w_k)$ be a sequence in $W(\mathbf{A})$ , with $w_k = f(\mathbf{x}_k)$ and $\left\| \mathbf{x}_k \right\| = 1$ . By the compactness of the unit sphere, fact (A3) of Chapter 16’s introduction, a subsequence $(\mathbf{x}_{k_l})$ converges to a unit vector $\mathbf{x}$ . By continuity, $w_{k_l} = f(\mathbf{x}_{k_l}) \to f(\mathbf{x}) \in W(\mathbf{A})$ . If $(w_k)$ itself converges to $w$ , its subsequence converges to $w$ too, and limits are unique, so $w = f(\mathbf{x}) \in W(\mathbf{A})$ : the set is closed.
(e) For a unit $\mathbf{x}$ , by the Cauchy–Schwarz inequality and Theorem 16.3.2 (a), $\lvert\mathbf{x}^{*}\mathbf{A}\mathbf{x}\rvert = \lvert\langle \mathbf{A}\mathbf{x}, \mathbf{x} \rangle\rvert \le \left\| \mathbf{A}\mathbf{x} \right\|\,\left\| \mathbf{x} \right\| \le \left\| \mathbf{A} \right\|_2$ .

Part (a) makes $W(\mathbf{A})$ an inclusion region for the spectrum, and (e) puts it inside the disc of radius $\left\| \mathbf{A} \right\|_2$ , which upgrades $\rho(\mathbf{A}) \le \left\| \mathbf{A} \right\|_2$ from the spectrum to the whole numerical range. Part (b) is the tool that lets us move any two points of $W(\mathbf{A})$ to $0$ and $1$ , and the next proof depends on it. Part (c) says $W(\mathbf{A})$ depends only on the operator $T_{\mathbf{A}}$ and the inner product, not on the orthonormal basis used to write it down.

20.2.3. Convexity: the Toeplitz–Hausdorff theorem

The examples so far were all convex: a point, a segment, a triangle, a disc. That is not a coincidence. To prove it, split $\mathbf{A}$ into two Hermitian pieces, the way a complex number splits into its real and imaginary parts.

Lemma 20.2.3 (Real and Imaginary Parts of a Matrix)

Let $\mathbf{A} \in M_n(\mathbb{C})$ , and put $\mathbf{H} = \frac{\mathbf{A} + \mathbf{A}^{*}}{2}, \qquad \mathbf{K} = \frac{\mathbf{A} - \mathbf{A}^{*}}{2i} .$ Then $\mathbf{H}$ and $\mathbf{K}$ are Hermitian, $\mathbf{A} = \mathbf{H} + i\mathbf{K}$ , and for every $\mathbf{x} \in \mathbb{C}^n$ the numbers $\mathbf{x}^{*}\mathbf{H}\mathbf{x}$ and $\mathbf{x}^{*}\mathbf{K}\mathbf{x}$ are real, with $\operatorname{Re}(\mathbf{x}^{*}\mathbf{A}\mathbf{x}) = \mathbf{x}^{*}\mathbf{H}\mathbf{x}, \qquad \operatorname{Im}(\mathbf{x}^{*}\mathbf{A}\mathbf{x}) = \mathbf{x}^{*}\mathbf{K}\mathbf{x} .$

Proof. By Definition 11.1.2, $(\mathbf{A}^{*})^{*} = \mathbf{A}$ , $(\mathbf{B} + \mathbf{C})^{*} = \mathbf{B}^{*} + \mathbf{C}^{*}$ , and $(c\mathbf{B})^{*} = \overline{c}\mathbf{B}^{*}$ . Hence $\mathbf{H}^{*} = \tfrac12(\mathbf{A}^{*} + \mathbf{A}) = \mathbf{H}$ and $\mathbf{K}^{*} = \tfrac{1}{-2i}(\mathbf{A}^{*} - \mathbf{A}) = \tfrac{1}{2i}(\mathbf{A} - \mathbf{A}^{*}) = \mathbf{K}$ . Also $\mathbf{H} + i\mathbf{K} = \tfrac12(\mathbf{A} + \mathbf{A}^{*}) + \tfrac12(\mathbf{A} - \mathbf{A}^{*}) = \mathbf{A}$ . For $\mathbf{x} \in \mathbb{C}^n$ , $\mathbf{x}^{*}\mathbf{H}\mathbf{x} = \langle \mathbf{H}\mathbf{x}, \mathbf{x} \rangle$ is real by Proposition 11.6.7 (a), applied to the self-adjoint operator $T_{\mathbf{H}}$ , and likewise $\mathbf{x}^{*}\mathbf{K}\mathbf{x}$ . So $\mathbf{x}^{*}\mathbf{A}\mathbf{x} = \mathbf{x}^{*}\mathbf{H}\mathbf{x} + i\,\mathbf{x}^{*}\mathbf{K}\mathbf{x}$ is the splitting of a complex number into real and imaginary parts.

For example $\mathbf{J}_2(0)$ has $\mathbf{H} = \tfrac12\begin{psmallmatrix} 0 & 1 \\ 1 & 0 \end{psmallmatrix}$ and $\mathbf{K} = \tfrac{1}{2i}\begin{psmallmatrix} 0 & 1 \\ -1 & 0 \end{psmallmatrix} = \tfrac12\begin{psmallmatrix} 0 & -i \\ i & 0 \end{psmallmatrix}$ , both with eigenvalues $\pm\tfrac12$ . A Hermitian $\mathbf{A}$ has $\mathbf{K} = \mathbf{O}$ , and then $W(\mathbf{A})$ lies on the real axis.

Theorem 20.2.4 (Toeplitz–Hausdorff Theorem)

For every $\mathbf{A} \in M_n(\mathbb{C})$ , the numerical range $W(\mathbf{A})$ is a convex subset of $\mathbb{C}$ .

Idea. Step roadmap. ① By Proposition 20.2.2 (b), an affine change $z \mapsto \alpha z + \beta$ carries any two points of $W(\mathbf{A})$ to $0$ and $1$ , and carries segments to segments; so it suffices to show that $[0, 1] \subseteq W(\mathbf{B})$ whenever $0, 1 \in W(\mathbf{B})$ . ② Take unit vectors $\mathbf{x}$ , $\mathbf{y}$ with $\mathbf{x}^{*}\mathbf{B}\mathbf{x} = 0$ and $\mathbf{y}^{*}\mathbf{B}\mathbf{y} = 1$ , and walk from $\mathbf{x}$ to $\mathbf{y}$ along the straight path $(1 - t)\mathbf{x} + t\mathbf{y}$ . Along the way the value of $\mathbf{B}$ , normalized, moves continuously from $0$ to $1$ , but it may wander off the real axis. ③ Its imaginary part along the path is $\mathbf{K}$ evaluated there, and expanding shows that only the cross term $\mathbf{x}^{*}\mathbf{K}\mathbf{y} + \mathbf{y}^{*}\mathbf{K}\mathbf{x} = 2\operatorname{Re}(\mathbf{x}^{*}\mathbf{K}\mathbf{y})$ can be nonzero. Multiplying $\mathbf{y}$ by a unimodular scalar rotates $\mathbf{x}^{*}\mathbf{K}\mathbf{y}$ freely without changing $\mathbf{y}^{*}\mathbf{B}\mathbf{y}$ , so we rotate it onto the imaginary axis, where its real part is $0$ . ④ Then the path stays on the real axis, and the intermediate value theorem fills in $[0, 1]$ .

Proof. Step 1: reduction. Let $w_0, w_1 \in W(\mathbf{A})$ and $0 \le s \le 1$ ; we must show $(1 - s)w_0 + sw_1 \in W(\mathbf{A})$ . If $w_0 = w_1$ there is nothing to prove. Otherwise put $\alpha = 1/(w_1 - w_0)$ , $\beta = -\alpha w_0$ , and $\mathbf{B} = \alpha\mathbf{A} + \beta\mathbf{I}$ . By Proposition 20.2.2 (b), $W(\mathbf{B}) = \alpha W(\mathbf{A}) + \beta$ , which contains $\alpha w_0 + \beta = 0$ and $\alpha w_1 + \beta = 1$ . Suppose we show $s \in W(\mathbf{B})$ . Then $s = \alpha w + \beta$ for some $w \in W(\mathbf{A})$ , and solving, $w = w_0 + s(w_1 - w_0) = (1 - s)w_0 + sw_1$ . So it suffices to prove: if $0, 1 \in W(\mathbf{B})$ , then $[0, 1] \subseteq W(\mathbf{B})$ .

Step 2: choosing the vectors. Let $\mathbf{B} = \mathbf{H} + i\mathbf{K}$ as in Lemma 20.2.3, and choose unit vectors $\mathbf{x}$ , $\mathbf{y}_0$ with $\mathbf{x}^{*}\mathbf{B}\mathbf{x} = 0$ and $\mathbf{y}_0^{*}\mathbf{B}\mathbf{y}_0 = 1$ . By Lemma 20.2.3, $\mathbf{x}^{*}\mathbf{H}\mathbf{x} = \mathbf{x}^{*}\mathbf{K}\mathbf{x} = 0$ , $\mathbf{y}_0^{*}\mathbf{H}\mathbf{y}_0 = 1$ , and $\mathbf{y}_0^{*}\mathbf{K}\mathbf{y}_0 = 0$ . Let $c = \mathbf{x}^{*}\mathbf{K}\mathbf{y}_0$ , and put $\gamma = 1$ if $c = 0$ , and $\gamma = i\overline{c}/\lvert c\rvert$ otherwise. In both cases $\lvert\gamma\rvert = 1$ and $\gamma c = i\lvert c\rvert$ is purely imaginary. Set $\mathbf{y} = \gamma\mathbf{y}_0$ . Then $\left\| \mathbf{y} \right\| = 1$ and $\mathbf{y}^{*}\mathbf{B}\mathbf{y} = \lvert\gamma\rvert^2\,\mathbf{y}_0^{*}\mathbf{B}\mathbf{y}_0 = 1$ , so still $\mathbf{y}^{*}\mathbf{H}\mathbf{y} = 1$ and $\mathbf{y}^{*}\mathbf{K}\mathbf{y} = 0$ . Since $\mathbf{K}$ is Hermitian, $\mathbf{y}^{*}\mathbf{K}\mathbf{x} = (\mathbf{x}^{*}\mathbf{K}\mathbf{y})^{*} = \overline{\mathbf{x}^{*}\mathbf{K}\mathbf{y}}$ , and so $\mathbf{x}^{*}\mathbf{K}\mathbf{y} + \mathbf{y}^{*}\mathbf{K}\mathbf{x} = 2\operatorname{Re}(\mathbf{x}^{*}\mathbf{K}\mathbf{y}) = 2\operatorname{Re}(\gamma c) = 0 .$ Finally, $\mathbf{x}$ and $\mathbf{y}$ are linearly independent. Otherwise, both being nonzero, $\mathbf{y} = \mu\mathbf{x}$ for a scalar $\mu$ , and then $1 = \mathbf{y}^{*}\mathbf{B}\mathbf{y} = \lvert\mu\rvert^2\,\mathbf{x}^{*}\mathbf{B}\mathbf{x} = 0$ , which is absurd.

Step 3: the path stays real. For $t \in [0, 1]$ put $\mathbf{v}(t) = (1 - t)\mathbf{x} + t\mathbf{y}$ . The coefficients $1 - t$ and $t$ are not both zero, so $\mathbf{v}(t) \ne \mathbf{0}$ by independence. Expanding, and using $\mathbf{x}^{*}\mathbf{K}\mathbf{x} = \mathbf{y}^{*}\mathbf{K}\mathbf{y} = 0$ and Step 2, $\mathbf{v}(t)^{*}\mathbf{K}\mathbf{v}(t) = (1 - t)^2\,\mathbf{x}^{*}\mathbf{K}\mathbf{x} + t^2\,\mathbf{y}^{*}\mathbf{K}\mathbf{y} + t(1 - t)\bigl(\mathbf{x}^{*}\mathbf{K}\mathbf{y} + \mathbf{y}^{*}\mathbf{K}\mathbf{x}\bigr) = 0 .$ So $\mathbf{v}(t)^{*}\mathbf{B}\mathbf{v}(t) = \mathbf{v}(t)^{*}\mathbf{H}\mathbf{v}(t)$ is real for every $t$ . The same expansion with $\mathbf{H}$ in place of $\mathbf{K}$ , and with $\left\| \mathbf{v}(t) \right\|^2 = \mathbf{v}(t)^{*}\mathbf{v}(t)$ , gives $\begin{aligned} \mathbf{v}(t)^{*}\mathbf{H}\mathbf{v}(t) &= t^2 + 2t(1 - t)\operatorname{Re}(\mathbf{x}^{*}\mathbf{H}\mathbf{y}) ,\\ \left\| \mathbf{v}(t) \right\|^2 &= (1 - t)^2 + t^2 + 2t(1 - t)\operatorname{Re}(\mathbf{x}^{*}\mathbf{y}) , \end{aligned}$ two polynomials in $t$ with real coefficients, the second positive on $[0, 1]$ .

Step 4: filling the segment. The function $f(t) = \frac{\mathbf{v}(t)^{*}\mathbf{H}\mathbf{v}(t)}{\left\| \mathbf{v}(t) \right\|^2}, \qquad 0 \le t \le 1 ,$ is a quotient of polynomials with a denominator that never vanishes, hence continuous by the algebra of limits, and $f(0) = \mathbf{x}^{*}\mathbf{H}\mathbf{x} = 0$ , $f(1) = \mathbf{y}^{*}\mathbf{H}\mathbf{y} = 1$ . Let $s \in [0, 1]$ . By the intermediate value theorem, fact (A5) of Chapter 16’s introduction, there is $t_s \in [0, 1]$ with $f(t_s) = s$ . The vector $\mathbf{u} = \mathbf{v}(t_s)/\left\| \mathbf{v}(t_s) \right\|$ is a unit vector, and by Step 3, $\mathbf{u}^{*}\mathbf{B}\mathbf{u} = \frac{\mathbf{v}(t_s)^{*}\mathbf{B}\mathbf{v}(t_s)}{\left\| \mathbf{v}(t_s) \right\|^2} = \frac{\mathbf{v}(t_s)^{*}\mathbf{H}\mathbf{v}(t_s)}{\left\| \mathbf{v}(t_s) \right\|^2} = f(t_s) = s .$ So $s \in W(\mathbf{B})$ . By Step 1, $W(\mathbf{A})$ contains the segment between any two of its points, which is Definition 18.1.1. This proves the theorem.

The phase choice in Step 2 is the heart of the proof. Without it the imaginary part along the path is $2t(1 - t)\operatorname{Re}(\mathbf{x}^{*}\mathbf{K}\mathbf{y}_0)/\left\| \mathbf{v}(t) \right\|^2$ , which may be nonzero, and the path would leave the real axis between $0$ and $1$ . The rotated vector $\gamma\mathbf{y}_0$ spans the same line as $\mathbf{y}_0$ and gives the same value of $\mathbf{B}$ , so the rotation costs nothing. Only the path between $\mathbf{x}$ and $\mathbf{y}$ changes.

With convexity, $W(\mathbf{A})$ contains the convex hull of $\operatorname{spec}(\mathbf{A})$ , by (a) and Definition 18.1.4, and it contains the diagonal entries, so it contains $\operatorname{conv}\{a_{11}, \dots, a_{nn}\}$ as well.

20.2.4. Normal matrices

For a diagonal matrix the numerical range is the convex hull of the diagonal entries, and those are the eigenvalues. Unitary similarity does not change $W$ , and a normal matrix is unitarily similar to a diagonal one. So for a normal matrix $W(\mathbf{A})$ is as small as (a) and convexity allow.

Proposition 20.2.5 (The Numerical Range of a Normal Matrix)

Let $\mathbf{A} \in M_n(\mathbb{C})$ be normal, with eigenvalues $\lambda_1, \dots, \lambda_n$ , listed with multiplicity. Then $W(\mathbf{A}) = \operatorname{conv}\{\lambda_1, \dots, \lambda_n\} .$

Idea. Unitary similarity leaves $W$ alone, by Proposition 20.2.2 (c), so we may replace $\mathbf{A}$ by a diagonal matrix of its eigenvalues. Then $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ is the average of the $\lambda_i$ with weights $\lvert x_i\rvert^2$ , and the two inclusions say exactly that the lists of admissible weights and of convex coefficients are the same.

Proof. By Corollary 12.4.3, $\mathbf{A} = \mathbf{U}\mathbf{D}\mathbf{U}^{*}$ with $\mathbf{U}$ unitary and $\mathbf{D} = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$ . So $\mathbf{U}^{*}\mathbf{A}\mathbf{U} = \mathbf{D}$ , and $W(\mathbf{A}) = W(\mathbf{D})$ by Proposition 20.2.2 (c). For a unit $\mathbf{x}$ , $\mathbf{x}^{*}\mathbf{D}\mathbf{x} = \sum_i\lvert x_i\rvert^2\lambda_i$ .

$(\subseteq)$ The weights $\lvert x_i\rvert^2$ are nonnegative with sum $1$ , so $\mathbf{x}^{*}\mathbf{D}\mathbf{x}$ is a convex combination of $\lambda_1, \dots, \lambda_n$ , and lies in their convex hull by Theorem 18.1.3.

$(\supseteq)$ By Theorem 18.1.3, a point of the hull is $\sum_{k=1}^{m}t_k\mu_k$ with $t_k \ge 0$ , $\sum_k t_k = 1$ , and each $\mu_k$ one of the $\lambda_i$ . Assign each $k$ to one index $i$ with $\lambda_i = \mu_k$ , and let $s_i$ be the sum of the $t_k$ assigned to $i$ . Then $s_i \ge 0$ , $\sum_i s_i = 1$ , and the point is $\sum_i s_i\lambda_i$ . The unit vector with $x_i = \sqrt{s_i}$ gives $\mathbf{x}^{*}\mathbf{D}\mathbf{x} = \sum_i s_i\lambda_i$ .

For a Hermitian matrix this recovers Proposition 17.1.1. For a unitary matrix, whose eigenvalues lie on the unit circle, $W(\mathbf{A})$ is the convex hull of those eigenvalues: a polygon inscribed in the circle when three or more of them are distinct, and a chord or a single point when there are two or one. For a nonnormal matrix $W(\mathbf{A})$ can be much larger than $\operatorname{conv}\operatorname{spec}(\mathbf{A})$ : by Example (The numerical range of a Jordan block), $W(\mathbf{J}_2(0))$ is a disc of radius $\tfrac12$ around the single eigenvalue $0$ . The extra room is a measure of nonnormality, and Exercise C3 turns it into a comparison with $\left\| \mathbf{A} \right\|_2$ .

Quick check. Find $W(\mathbf{A})$ for the rotation $\mathbf{A} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ , regarded as a complex matrix.

Solution. $\mathbf{A}^{\top}\mathbf{A} = \mathbf{I}$ , so $\mathbf{A}$ is real orthogonal, hence unitary and normal. Its characteristic polynomial is $x^2 + 1$ , with roots $\pm i$ . By Proposition 20.2.5, $W(\mathbf{A})$ is the segment from $-i$ to $i$ on the imaginary axis. For real unit vectors $\mathbf{x}$ the value $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ is $0$ , which is why the real Rayleigh-type quotient of Chapter 17 §01 saw nothing; the complex unit vectors fill in the rest.

20.2.5. Bendixson’s rectangle

By Lemma 20.2.3, the real part of $\mathbf{x}^{*}\mathbf{A}\mathbf{x}$ is a value of the Hermitian form of $\mathbf{H}$ , and Proposition 17.1.1 confines those values to $[\lambda_n(\mathbf{H}), \lambda_1(\mathbf{H})]$ . The same holds for the imaginary part and $\mathbf{K}$ . Together they box in the whole numerical range, hence the spectrum.

Theorem 20.2.6 (Bendixson’s Theorem)

Let $\mathbf{A} \in M_n(\mathbb{C})$ , with $\mathbf{H} = (\mathbf{A} + \mathbf{A}^{*})/2$ and $\mathbf{K} = (\mathbf{A} - \mathbf{A}^{*})/(2i)$ . Every $w \in W(\mathbf{A})$ satisfies $\begin{aligned} \lambda_n(\mathbf{H}) &\le \operatorname{Re}w \le \lambda_1(\mathbf{H}) ,\\ \lambda_n(\mathbf{K}) &\le \operatorname{Im}w \le \lambda_1(\mathbf{K}) . \end{aligned}$ In particular every eigenvalue $\lambda$ of $\mathbf{A}$ satisfies the same inequalities with $\lambda$ in place of $w$ .

Proof. Let $w = \mathbf{x}^{*}\mathbf{A}\mathbf{x}$ with $\left\| \mathbf{x} \right\| = 1$ . By Lemma 20.2.3, $\operatorname{Re}w = \mathbf{x}^{*}\mathbf{H}\mathbf{x} = \langle \mathbf{H}\mathbf{x}, \mathbf{x} \rangle$ and $\operatorname{Im}w = \mathbf{x}^{*}\mathbf{K}\mathbf{x}$ , and $\mathbf{H}$ , $\mathbf{K}$ are Hermitian. By Lemma 13.5.2, $\lambda_n(\mathbf{H}) \le \langle \mathbf{H}\mathbf{x}, \mathbf{x} \rangle \le \lambda_1(\mathbf{H})$ , and likewise for $\mathbf{K}$ . Every eigenvalue lies in $W(\mathbf{A})$ by Proposition 20.2.2 (a).

The most used consequence concerns the sign of the real parts. If $\mathbf{A} + \mathbf{A}^{*} \succ \mathbf{O}$ , then $\lambda_n(\mathbf{H}) > 0$ , so every eigenvalue of $\mathbf{A}$ has positive real part, and in particular $\mathbf{A}$ is invertible. This needs nothing about the size of the off-diagonal entries, only about their symmetric part.

Example (A rectangle beats the discs)

Let $\mathbf{A} = \begin{pmatrix} 3 & 2 \\ -4 & 1 \end{pmatrix}$ . Show that every eigenvalue of $\mathbf{A}$ has positive real part, using Theorem 20.2.6, and check that the Gershgorin discs cannot show it. Compare with the eigenvalues.

Solution. Here $\mathbf{A}^{*} = \mathbf{A}^{\top}$ , so $\mathbf{H} = \begin{pmatrix} 3 & -1 \\ -1 & 1 \end{pmatrix}, \qquad \mathbf{K} = \frac{1}{2i}\begin{pmatrix} 0 & 6 \\ -6 & 0 \end{pmatrix} = \begin{pmatrix} 0 & -3i \\ 3i & 0 \end{pmatrix} .$ $\mathbf{H}$ has trace $4$ and determinant $2$ , so $p_{\mathbf{H}}(x) = x^2 - 4x + 2$ , with roots $2 \pm \sqrt2$ . $\mathbf{K}$ has trace $0$ and determinant $0 - (-3i)(3i) = -9$ , so its eigenvalues are $\pm 3$ . By Theorem 20.2.6, every eigenvalue lies in the rectangle $2 - \sqrt2 \le \operatorname{Re}\lambda \le 2 + \sqrt2, \qquad -3 \le \operatorname{Im}\lambda \le 3 ,$ and in particular $\operatorname{Re}\lambda \ge 2 - \sqrt2 > 0$ .

The row discs are $\lvert z - 3\rvert \le 2$ and $\lvert z - 1\rvert \le 4$ , and the second contains $-3$ . The column discs are $\lvert z - 3\rvert \le 4$ and $\lvert z - 1\rvert \le 2$ , and the first contains $-1$ . So neither union rules out an eigenvalue with negative real part.

Directly, $p_{\mathbf{A}}(x) = x^2 - 4x + 11$ , so $\lambda = 2 \pm i\sqrt7$ . The real part $2$ lies in $[2 - \sqrt2, 2 + \sqrt2]$ , and the imaginary parts $\pm\sqrt7 \approx \pm 2.65$ lie in $[-3, 3]$ .

20.2.6. What these regions do not do

Every region in this section and the last is an inclusion region: it contains the spectrum. None of them is a perturbation bound in the sense of the rest of this chapter, which asks how far the eigenvalues move when $\mathbf{A}$ changes by a small $\mathbf{E}$ . The numerical range does give a statement of that kind. For a unit $\mathbf{x}$ , $\mathbf{x}^{*}(\mathbf{A} + \mathbf{E})\mathbf{x} = \mathbf{x}^{*}\mathbf{A}\mathbf{x} + \mathbf{x}^{*}\mathbf{E}\mathbf{x}$ , and $\lvert\mathbf{x}^{*}\mathbf{E}\mathbf{x}\rvert \le \left\| \mathbf{E} \right\|_2$ by Proposition 20.2.2 (e). Hence $W(\mathbf{A} + \mathbf{E}) \subseteq W(\mathbf{A}) + W(\mathbf{E}) \subseteq \{\, w + e : w \in W(\mathbf{A}),\ \lvert e\rvert \le \left\| \mathbf{E} \right\|_2 \,\} ,$ and so every eigenvalue of $\mathbf{A} + \mathbf{E}$ lies within $\left\| \mathbf{E} \right\|_2$ of $W(\mathbf{A})$ . This is only as good as $W(\mathbf{A})$ is small.

Warning. $W(\mathbf{A})$ can be far larger than $\operatorname{spec}(\mathbf{A})$ , so “within $\left\| \mathbf{E} \right\|_2$ of $W(\mathbf{A})$ ” is not “within $\left\| \mathbf{E} \right\|_2$ of an eigenvalue”. For $\mathbf{A} = M\mathbf{J}_2(0)$ with $M > 0$ , Proposition 20.2.2 (b) and Example (The numerical range of a Jordan block) give $W(\mathbf{A}) = \{\lvert z\rvert \le M/2\}$ , while $\operatorname{spec}(\mathbf{A}) = \{0\}$ . So the bound above allows the eigenvalues of $\mathbf{A} + \mathbf{E}$ to lie anywhere in $\lvert z\rvert \le M/2 + \left\| \mathbf{E} \right\|_2$ , and this does not shrink as $\mathbf{E} \to \mathbf{O}$ . The eigenvalues do move a long way: $\mathbf{A} + \varepsilon\mathbf{e}_2\mathbf{e}_1^{\top}$ has characteristic polynomial $x^2 - M\varepsilon$ , so for $\varepsilon > 0$ its eigenvalues $\pm\sqrt{M\varepsilon}$ have moved by $\sqrt{M\varepsilon}$ , far more than $\varepsilon$ . For a normal $\mathbf{A}$ , $W(\mathbf{A}) = \operatorname{conv}\operatorname{spec}(\mathbf{A})$ is small, but it still contains points far from every eigenvalue, such as the midpoint of two distant ones.

The genuine perturbation bounds need a different mechanism: §03 proves that eigenvalues move continuously and at what rate in general, and §04 gives Bauer–Fike’s bound, in which the distance to the nearest eigenvalue is controlled.

20.2.7. Exercises

A. Check your understanding

Exercise (A1)

(a) Define the Brauer oval $O_{ij}(\mathbf{A})$ and the numerical range $W(\mathbf{A})$ , and state the Toeplitz–Hausdorff theorem.
(b) True or false: $W(\mathbf{A}) = \operatorname{conv}\operatorname{spec}(\mathbf{A})$ for every $\mathbf{A} \in M_n(\mathbb{C})$ . Justify your answer.
(c) True or false: $W(\mathbf{A})$ is contained in the union of the Gershgorin discs of $\mathbf{A}$ . Justify your answer.
(d) True or false: if $\mathbf{A} + \mathbf{A}^{*} \succ \mathbf{O}$ , then $\mathbf{A}$ is invertible. Justify your answer.
(e) Why does Theorem 20.2.1 require $n \ge 2$ ?

Solution.

(a) For $i \ne j$ , $O_{ij}(\mathbf{A}) = \{z : \lvert z - a_{ii}\rvert\lvert z - a_{jj}\rvert \le r_i(\mathbf{A})r_j(\mathbf{A})\}$ , and $W(\mathbf{A}) = \{\mathbf{x}^{*}\mathbf{A}\mathbf{x} : \left\| \mathbf{x} \right\| = 1\}$ . Theorem 20.2.4: for every square complex $\mathbf{A}$ , $W(\mathbf{A})$ is convex.
(b) False. $W(\mathbf{J}_2(0))$ is the disc $\lvert z\rvert \le \tfrac12$ (Example (The numerical range of a Jordan block)), while $\operatorname{conv}\operatorname{spec}(\mathbf{J}_2(0)) = \{0\}$ . Equality holds for normal matrices (Proposition 20.2.5).
(c) False. For $\mathbf{A} = \operatorname{diag}(1, -1)$ the discs are the two points $\{1\}$ and $\{-1\}$ , while $W(\mathbf{A}) = [-1, 1]$ contains $0$ . The discs contain the spectrum, not the numerical range.
(d) True. By Theorem 20.2.6 every eigenvalue has $\operatorname{Re}\lambda \ge \lambda_n(\mathbf{H}) > 0$ , with $\mathbf{H} = (\mathbf{A} + \mathbf{A}^{*})/2 \succ \mathbf{O}$ . So $0$ is not an eigenvalue, and $\mathbf{A}$ is invertible (Theorem 9.2.1).
(e) An oval is built from two distinct indices $i \ne j$ , and the proof needs a second index $q \ne p$ . For $n = 1$ there are no ovals, and the union in (a) would be empty, although the matrix has an eigenvalue.

B. Practice

Exercise (B1: Invertibility from ovals)

Let $\mathbf{A} = \begin{pmatrix} 5 & 1 & 1 \\ 1 & 5 & 1 \\ 2 & 2 & 2 \end{pmatrix}$ .

(a) Show that neither the row discs nor the column discs of $\mathbf{A}$ exclude $0$ .
(b) Use Theorem 20.2.1 to show that $\mathbf{A}$ is invertible.
(c) Check that $(1, -1, 0)$ is an eigenvector, and verify that its eigenvalue lies in $O_{12}(\mathbf{A})$ .

Solution.

(a) The row radii are $2, 2, 4$ , and $D_3(\mathbf{A}) = \{\lvert z - 2\rvert \le 4\}$ contains $0$ . The column radii are $3, 3, 2$ , and the third column disc $\lvert z - 2\rvert \le 2$ contains $0$ .
(b) At $z = 0$ : for $O_{12}$ , $5 \cdot 5 = 25 > 4 = r_1r_2$ ; for $O_{13}$ and $O_{23}$ , $5 \cdot 2 = 10 > 8 = r_1r_3 = r_2r_3$ . So $0$ lies in no oval, is not an eigenvalue by Theorem 20.2.1 (a), and $\mathbf{A}$ is invertible. (Its determinant is $32$ .)
(c) $\mathbf{A}(1, -1, 0) = (5 - 1, 1 - 5, 2 - 2) = (4, -4, 0)$ , so the eigenvalue is $4$ . For $O_{12}$ : $\lvert 4 - 5\rvert\lvert 4 - 5\rvert = 1 \le 4$ .

Exercise (B2: A Bendixson rectangle)

Let $\mathbf{A} = \begin{pmatrix} 1 & 3 \\ -1 & 1 \end{pmatrix}$ . Find the rectangle of Theorem 20.2.6, and hence show that every eigenvalue of $\mathbf{A}$ satisfies $0 \le \operatorname{Re}\lambda \le 2$ and $\lvert\operatorname{Im}\lambda\rvert \le 2$ . Compare with the eigenvalues.

Solution. $\mathbf{H} = \tfrac12(\mathbf{A} + \mathbf{A}^{\top}) = \begin{psmallmatrix} 1 & 1 \\ 1 & 1 \end{psmallmatrix}$ , with $p_{\mathbf{H}}(x) = x^2 - 2x$ , so its eigenvalues are $0$ and $2$ . $\mathbf{K} = \tfrac{1}{2i}(\mathbf{A} - \mathbf{A}^{\top}) = \tfrac{1}{2i}\begin{psmallmatrix} 0 & 4 \\ -4 & 0 \end{psmallmatrix} = \begin{psmallmatrix} 0 & -2i \\ 2i & 0 \end{psmallmatrix}$ , with trace $0$ and determinant $-4$ , so its eigenvalues are $\pm 2$ . By Theorem 20.2.6 every eigenvalue has $0 \le \operatorname{Re}\lambda \le 2$ and $-2 \le \operatorname{Im}\lambda \le 2$ . Directly, $p_{\mathbf{A}}(x) = x^2 - 2x + 4$ , so $\lambda = 1 \pm i\sqrt3$ : the real part $1$ lies in $[0, 2]$ , and $\sqrt3 \approx 1.73 \le 2$ .

Exercise (B3: Computing numerical ranges)

Determine $W(\mathbf{A})$ for each matrix. Justify your answers.

(a) $\mathbf{A} = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}$
(b) $\mathbf{A} = \operatorname{diag}(1, i, -1, -i)$
(c) $\mathbf{A} = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$

Solution.

(a) $\mathbf{A} = 2\mathbf{J}_2(0) + \mathbf{I}$ , so by Proposition 20.2.2 (b) and Example (The numerical range of a Jordan block), $W(\mathbf{A}) = 2W(\mathbf{J}_2(0)) + 1 = \{z : \lvert z - 1\rvert \le 1\}$ , the closed disc of radius $1$ about $1$ .
(b) A diagonal matrix is normal, so by Proposition 20.2.5 $W(\mathbf{A}) = \operatorname{conv}\{1, i, -1, -i\}$ , the filled square with these four vertices.
(c) $\mathbf{A}$ is real symmetric, with eigenvalues $3$ and $1$ (eigenvectors $(1, 1)$ and $(1, -1)$ ). By Proposition 17.1.1, or Proposition 20.2.5, $W(\mathbf{A}) = [1, 3]$ .

C. Going deeper

Exercise (C1: Direct sums)

(a) Let $\mathbf{B} \in M_k(\mathbb{C})$ and $\mathbf{C} \in M_m(\mathbb{C})$ . Prove that $W(\mathbf{B} \oplus \mathbf{C}) = \operatorname{conv}\bigl(W(\mathbf{B}) \cup W(\mathbf{C})\bigr)$ .
(b) Deduce that $\mathbf{A} = \operatorname{diag}(2, -2, 2i, -2i) \oplus \mathbf{J}_2(0)$ is not normal, but satisfies $W(\mathbf{A}) = \operatorname{conv}\operatorname{spec}(\mathbf{A})$ .

Hint for (a): one inclusion needs Theorem 20.2.4.

Solution.

(a) $(\subseteq)$ Write a unit vector of $\mathbb{C}^{k+m}$ as $\mathbf{x} = (\mathbf{y}, \mathbf{z})$ , with $\left\| \mathbf{y} \right\|^2 + \left\| \mathbf{z} \right\|^2 = 1$ . Then $\mathbf{x}^{*}(\mathbf{B} \oplus \mathbf{C})\mathbf{x} = \mathbf{y}^{*}\mathbf{B}\mathbf{y} + \mathbf{z}^{*}\mathbf{C}\mathbf{z}$ . If $\mathbf{z} = \mathbf{0}$ this is a point of $W(\mathbf{B})$ , and if $\mathbf{y} = \mathbf{0}$ a point of $W(\mathbf{C})$ . Otherwise, with $s = \left\| \mathbf{y} \right\|^2 \in (0, 1)$ , $\mathbf{x}^{*}(\mathbf{B} \oplus \mathbf{C})\mathbf{x} = s\,\frac{\mathbf{y}^{*}\mathbf{B}\mathbf{y}}{\left\| \mathbf{y} \right\|^2} + (1 - s)\,\frac{\mathbf{z}^{*}\mathbf{C}\mathbf{z}}{\left\| \mathbf{z} \right\|^2} ,$ a convex combination of a point of $W(\mathbf{B})$ , at the unit vector $\mathbf{y}/\left\| \mathbf{y} \right\|$ , and a point of $W(\mathbf{C})$ . In every case the value lies in $\operatorname{conv}(W(\mathbf{B}) \cup W(\mathbf{C}))$ , by Theorem 18.1.3.
$(\supseteq)$ Taking $\mathbf{x} = (\mathbf{y}, \mathbf{0})$ with $\mathbf{y}$ a unit vector shows $W(\mathbf{B}) \subseteq W(\mathbf{B} \oplus \mathbf{C})$ , and likewise $W(\mathbf{C}) \subseteq W(\mathbf{B} \oplus \mathbf{C})$ . By Theorem 20.2.4, $W(\mathbf{B} \oplus \mathbf{C})$ is convex, so it contains the smallest convex set containing $W(\mathbf{B}) \cup W(\mathbf{C})$ (Definition 18.1.4).
(b) $\mathbf{A}$ is not normal: its lower-right block is $\mathbf{J}_2(0)$ , and $\mathbf{A}^{*}\mathbf{A}$ and $\mathbf{A}\mathbf{A}^{*}$ differ in that block, since $\mathbf{J}_2(0)^{*}\mathbf{J}_2(0) = \operatorname{diag}(0, 1)$ and $\mathbf{J}_2(0)\mathbf{J}_2(0)^{*} = \operatorname{diag}(1, 0)$ . By (a), Proposition 20.2.5, and Example (The numerical range of a Jordan block), $W(\mathbf{A})$ is the convex hull of the square $Q = \operatorname{conv}\{\pm 2, \pm 2i\}$ and the disc $\{\lvert z\rvert \le \tfrac12\}$ . The square contains the disc: $Q = \{ z : \lvert\operatorname{Re}z\rvert + \lvert\operatorname{Im}z\rvert \le 2 \}$ , and $\lvert\operatorname{Re}z\rvert + \lvert\operatorname{Im}z\rvert \le 2\lvert z\rvert \le 1$ on the disc. So $W(\mathbf{A}) = Q$ . The spectrum is $\{\pm 2, \pm 2i, 0\}$ , whose hull is also $Q$ , since $0 \in Q$ . Hence $W(\mathbf{A}) = \operatorname{conv}\operatorname{spec}(\mathbf{A})$ , although $\mathbf{A}$ is not normal: the converse of Proposition 20.2.5 fails.

Exercise (C2: When the numerical range is real, or a point)

Let $\mathbf{A} \in M_n(\mathbb{C})$ .

(a) Prove that $W(\mathbf{A}) \subseteq \mathbb{R}$ if and only if $\mathbf{A}$ is Hermitian.
(b) Deduce that $W(\mathbf{A}) = \{c\}$ for a single $c \in \mathbb{C}$ if and only if $\mathbf{A} = c\mathbf{I}$ .

Solution.

(a) $(\Leftarrow)$ If $\mathbf{A}$ is Hermitian, $\mathbf{x}^{*}\mathbf{A}\mathbf{x} = \langle \mathbf{A}\mathbf{x}, \mathbf{x} \rangle$ is real by Proposition 11.6.7 (a). $(\Rightarrow)$ Write $\mathbf{A} = \mathbf{H} + i\mathbf{K}$ as in Lemma 20.2.3. If $W(\mathbf{A}) \subseteq \mathbb{R}$ , then $\mathbf{x}^{*}\mathbf{K}\mathbf{x} = \operatorname{Im}(\mathbf{x}^{*}\mathbf{A}\mathbf{x}) = 0$ for every unit $\mathbf{x}$ . By Proposition 17.1.1, $\lambda_1(\mathbf{K})$ and $\lambda_n(\mathbf{K})$ are values of $\mathbf{x}^{*}\mathbf{K}\mathbf{x}$ on unit vectors, so both are $0$ , and every eigenvalue of $\mathbf{K}$ is $0$ . By Corollary 12.4.3, $\mathbf{K} = \mathbf{U}\,\mathbf{O}\,\mathbf{U}^{*} = \mathbf{O}$ . Hence $\mathbf{A} = \mathbf{H}$ is Hermitian.
(b) $(\Leftarrow)$ is the example $W(c\mathbf{I}) = \{c\}$ . $(\Rightarrow)$ By Proposition 20.2.2 (b), $W(\mathbf{A} - c\mathbf{I}) = W(\mathbf{A}) - c = \{0\} \subseteq \mathbb{R}$ , so $\mathbf{A} - c\mathbf{I}$ is Hermitian by (a). By Proposition 17.1.1 its extreme eigenvalues are values of the form on unit vectors, hence both $0$ , and as in (a) the Hermitian matrix $\mathbf{A} - c\mathbf{I}$ with all eigenvalues $0$ is $\mathbf{O}$ .

Exercise (C3: The numerical radius)

For $\mathbf{A} \in M_n(\mathbb{C})$ , the numerical radius is $w(\mathbf{A}) = \max\{\lvert z\rvert : z \in W(\mathbf{A})\}$ .

(a) Explain why the maximum exists, and prove that $\rho(\mathbf{A}) \le w(\mathbf{A}) \le \left\| \mathbf{A} \right\|_2$ .
(b) Prove that $\left\| \mathbf{A} \right\|_2 \le 2\,w(\mathbf{A})$ .
(c) Show that both constants are attained: $w(\mathbf{A}) = \left\| \mathbf{A} \right\|_2$ for every normal $\mathbf{A}$ , and $\left\| \mathbf{A} \right\|_2 = 2\,w(\mathbf{A})$ for $\mathbf{A} = \mathbf{J}_2(0)$ .

Hint for (b): bound $\left\| \mathbf{H} \right\|_2$ and $\left\| \mathbf{K} \right\|_2$ separately.

Solution.

(a) $W(\mathbf{A})$ is nonempty and compact (Proposition 20.2.2 (d)), and $z \mapsto \lvert z\rvert$ is continuous, so the maximum is attained by the extreme value theorem, fact (A4) of Chapter 16’s introduction. Every eigenvalue lies in $W(\mathbf{A})$ , by Proposition 20.2.2 (a), so $\rho(\mathbf{A}) \le w(\mathbf{A})$ ; and $w(\mathbf{A}) \le \left\| \mathbf{A} \right\|_2$ by Proposition 20.2.2 (e).
(b) Let $\mathbf{A} = \mathbf{H} + i\mathbf{K}$ as in Lemma 20.2.3. By Lemma 17.3.5, $\left\| \mathbf{H} \right\|_2$ is the larger of $\lambda_1(\mathbf{H})$ and $-\lambda_n(\mathbf{H})$ . By Proposition 17.1.1 there is a unit $\mathbf{x}$ with $\lambda_1(\mathbf{H}) = \mathbf{x}^{*}\mathbf{H}\mathbf{x} = \operatorname{Re}(\mathbf{x}^{*}\mathbf{A}\mathbf{x}) \le \lvert\mathbf{x}^{*}\mathbf{A}\mathbf{x}\rvert \le w(\mathbf{A}) ,$ and a unit $\mathbf{y}$ with $-\lambda_n(\mathbf{H}) = -\operatorname{Re}(\mathbf{y}^{*}\mathbf{A}\mathbf{y}) \le w(\mathbf{A})$ . So $\left\| \mathbf{H} \right\|_2 \le w(\mathbf{A})$ , and the same argument with imaginary parts gives $\left\| \mathbf{K} \right\|_2 \le w(\mathbf{A})$ . As the operator norm is a norm (Theorem 16.3.2 (b)), $\left\| \mathbf{A} \right\|_2 \le \left\| \mathbf{H} \right\|_2 + \left\| i\mathbf{K} \right\|_2 \le 2\,w(\mathbf{A})$ .
(c) For normal $\mathbf{A}$ , $\rho(\mathbf{A}) = \left\| \mathbf{A} \right\|_2$ , as Chapter 16 §04 showed after Definition 16.4.1, so the chain in (a) collapses to equalities. For $\mathbf{J}_2(0)$ , Example (The numerical range of a Jordan block) gives $w(\mathbf{J}_2(0)) = \tfrac12$ , and $\left\| \mathbf{J}_2(0) \right\|_2 = 1$ , since $\left\| \mathbf{J}_2(0)\mathbf{x} \right\| = \lvert x_2\rvert \le \left\| \mathbf{x} \right\|$ with equality at $\mathbf{x} = \mathbf{e}_2$ . So $\left\| \mathbf{J}_2(0) \right\|_2 = 2\,w(\mathbf{J}_2(0))$ .