16.6 Bounds for the Exponential, and Two Derivatives

Chapter 10 §09 built $e^{\mathbf{A}}$ out of a series and closed with a promise: “Chapter 16 will return to $e^{\mathbf{A}}$ with norms available, which give quantitative bounds instead of exact formulas.” That is what this section does. It does not rebuild the exponential; Definition 10.9.2 and Theorem 10.9.3 stand as they are, and the Jordan form is not touched. What is new is that the series now has a size, so its sum has a ceiling, its truncation has an error bound, and its behavior as $t \to \infty$ has a rate.

The second half takes the other step Chapter 10 could not: it differentiates. The derivative of the determinant and the derivative of the inverse are both needed later in the book, and both require one fact from calculus that linear algebra does not supply. That fact is stated before it is used.

Throughout, $\left\| \cdot \right\|$ is the operator norm on $M_n(\mathbb{C})$ induced by a norm on $\mathbb{C}^{n}$ (Definition 16.3.2), so $\left\| \mathbf{I} \right\| = 1$ and $\left\| \mathbf{X}\mathbf{Y} \right\| \le \left\| \mathbf{X} \right\|\left\| \mathbf{Y} \right\|$ (Theorem 16.3.2 (c), (d)).

16.6.1. How big is the exponential

Chapter 10 §10’s criteria answered whether; the norm answers how much. The bound to aim at is the one the scalar case suggests: $\lvert e^{z}\rvert \le e^{\lvert z\rvert}$ , because the series has nonnegative terms once every term is replaced by its modulus. The matrix proof is the same sentence with submultiplicativity in place of $\lvert zw\rvert = \lvert z\rvert\lvert w\rvert$ .

One scalar fact is needed and is not on the chapter’s list of imports. We name it here rather than let it in quietly.

Remark. A quoted fact. For every real $s \ge 0$ the series $\sum_{m \ge 0}s^{m}/m!$ converges, with sum $e^{s}$ . This is fact (A1) of Chapter 10 §09, quoted there and quoted again here; this book does not prove it. Because its terms are nonnegative, its partial sums increase to $e^{s}$ , so every partial sum is at most $e^{s}$ — that last step is (A2), monotone convergence, from the list in this chapter’s introduction.

Theorem 16.6.1 (Bounds for the Exponential)

Let $\mathbf{A} \in M_n(\mathbb{C})$ and write $a = \left\| \mathbf{A} \right\|$ . Then

(a) $\left\| e^{\mathbf{A}} \right\| \le e^{a}$ ;
(b) $\left\| e^{\mathbf{A}} - \mathbf{I} \right\| \le e^{a} - 1$ ;
(c) $\left\| e^{\mathbf{A}} - \mathbf{I} - \mathbf{A} \right\| \le e^{a} - 1 - a \le \dfrac{a^{2}e^{a}}{2}$ .

In particular $\left\| e^{t\mathbf{A}} \right\| \le e^{\lvert t\rvert a}$ for every real $t$ .

Idea. All three are the same computation stopped at a different place: bound the norm of a partial sum by the corresponding partial sum of $\sum s^{m}/m!$ , then let the partial sums converge. The only thing that needs care is the passage to the limit, and Lemma 16.1.1 does it, exactly as in Theorem 16.5.1. For the second inequality in (c), compare $m!$ with $2\,(m-2)!$ .

Proof. Write $\mathbf{S}_M = \sum_{m=0}^{M}\mathbf{A}^{m}/m!$ . By Theorem 10.9.3 the sequence $(\mathbf{S}_M)$ converges entrywise to $e^{\mathbf{A}}$ , hence $\left\| \mathbf{S}_M - e^{\mathbf{A}} \right\| \to 0$ by Corollary 16.2.4, hence $\left\| \mathbf{S}_M \right\| \to \left\| e^{\mathbf{A}} \right\|$ by Lemma 16.1.1. The same three steps apply to $\mathbf{S}_M - \mathbf{I}$ and to $\mathbf{S}_M - \mathbf{I} - \mathbf{A}$ . So in each part it suffices to bound the partial sums, uniformly in $M$ .

(a) Submultiplicativity gives $\left\| \mathbf{A}^{m} \right\| \le a^{m}$ for $m \ge 1$ , by induction, and $\left\| \mathbf{A}^{0} \right\| = \left\| \mathbf{I} \right\| = 1$ . By subadditivity and the quoted fact above, $\left\| \mathbf{S}_M \right\| \le \sum_{m=0}^{M}\frac{\left\| \mathbf{A}^{m} \right\|}{m!} \le \sum_{m=0}^{M}\frac{a^{m}}{m!} \le e^{a} .$ Letting $M \to \infty$ gives $\left\| e^{\mathbf{A}} \right\| \le e^{a}$ .
(b) The same estimate with the term $m = 0$ removed: $\left\| \mathbf{S}_M - \mathbf{I} \right\| \le \sum_{m=1}^{M}\frac{a^{m}}{m!} \le e^{a} - 1 ,$ since the partial sums of $\sum_{m \ge 1}a^{m}/m!$ increase to $e^{a} - 1$ .
(c) Removing the terms $m = 0$ and $m = 1$ in the same way gives $\left\| e^{\mathbf{A}} - \mathbf{I} - \mathbf{A} \right\| \le \sum_{m \ge 2}\frac{a^{m}}{m!} = e^{a} - 1 - a .$ For the second inequality, note that $m! = m(m-1)\,(m-2)! \ge 2\,(m-2)!$ for every $m \ge 2$ , so $a^{m}/m! \le \tfrac12 a^{2}\cdot a^{m-2}/(m-2)!$ . Summing over $m \ge 2$ and substituting $l = m - 2$ , $\sum_{m \ge 2}\frac{a^{m}}{m!} \le \frac{a^{2}}{2}\sum_{l \ge 0}\frac{a^{l}}{l!} = \frac{a^{2}e^{a}}{2} .$

The final claim is (a) applied to $t\mathbf{A}$ , whose norm is $\lvert t\rvert a$ by homogeneity. This proves the theorem.

Part (c) is the statement that $e^{\mathbf{A}}$ agrees with $\mathbf{I} + \mathbf{A}$ to second order in $\left\| \mathbf{A} \right\|$ , with an explicit constant. It is the bound one uses when $\mathbf{A}$ is a small perturbation, and it is the counterpart, for the exponential, of Theorem 16.5.1 (c).

Example (Where the bound is exact, and where it is useless)

Compare $\left\| e^{\mathbf{A}} \right\|_2$ with $e^{\left\| \mathbf{A} \right\|_2}$ for

(a) $\mathbf{A} = \operatorname{diag}(2, 1)$ ;
(b) $\mathbf{A} = \begin{pmatrix} 0 & 1 \\ -1 & 0\end{pmatrix}$ , and more generally $t\mathbf{A}$ for $t > 0$ .

Solution.

(a) $\mathbf{A}$ is Hermitian, hence normal, so its singular values are the moduli of its eigenvalues and $\left\| \mathbf{A} \right\|_2 = 2$ by Theorem 16.3.3 (c). By Definition 10.9.1 (or directly from the series, entry by entry) $e^{\mathbf{A}} = \operatorname{diag}(e^{2}, e)$ , again normal, so $\left\| e^{\mathbf{A}} \right\|_2 = e^{2}$ . The bound $e^{\left\| \mathbf{A} \right\|_2} = e^{2}$ is attained exactly. The same happens for every $\mathbf{A} = c\mathbf{I}$ with $c > 0$ , so no constant smaller than $1$ can be inserted in Theorem 16.6.1 (a).
(b) $\mathbf{A}^{2} = -\mathbf{I}$ , so $\mathbf{A}^{3} = -\mathbf{A}$ , $\mathbf{A}^{4} = \mathbf{I}$ , and grouping the series by parity gives, using fact (A1) of Chapter 10 §09 for the series of $\cos$ and $\sin$ and its (A3) to rearrange a convergent series into its even and odd parts, $e^{t\mathbf{A}} = (\cos t)\mathbf{I} + (\sin t)\mathbf{A} = \begin{pmatrix} \cos t & \sin t \\ -\sin t & \cos t\end{pmatrix},$ a real orthogonal matrix. Hence $\left\| e^{t\mathbf{A}} \right\|_2 = 1$ for every $t$ , while $\left\| t\mathbf{A} \right\|_2 = t$ and the bound reads $e^{t}$ . At $t = 10$ the bound is about $22026$ and the truth is $1$ .

Warning. The bound sees only the size of $\mathbf{A}$ , never the direction of its spectrum. Part (b) above is the extreme case: $\left\| e^{t\mathbf{A}} \right\|$ stays at $1$ while $e^{t\left\| \mathbf{A} \right\|}$ runs away. The reason is that Theorem 16.6.1 throws away every cancellation in the series at the first step, when $\left\| \mathbf{A}^{m} \right\|$ is replaced by $\left\| \mathbf{A} \right\|^{m}$ . Any statement about $e^{t\mathbf{A}}$ for large $t$ must come from the eigenvalues instead, which is the subject of the next subsection.

Quick check. Why does Theorem 16.6.1 (a) fail for the Frobenius norm, and what is the correct statement for it?

Solution. Because $\left\| \mathbf{I}_n \right\|_F = \sqrt n$ , so already at $\mathbf{A} = \mathbf{O}$ the claim reads $\sqrt n \le 1$ . Repeating the proof with the term $m = 0$ kept separate gives the correct version for any matrix norm: $\left\| e^{\mathbf{A}} \right\| \le \left\| \mathbf{I} \right\| + e^{\left\| \mathbf{A} \right\|} - 1$ . This is the same correction as in Exercise (C3: Without $\left\| \mathbf{I} \right\| = 1$ ), and it comes from the same place: only an induced norm has $\left\| \mathbf{I} \right\| = 1$ .

16.6.2. Decay in the stable case

The warning above says the bound $e^{t\left\| \mathbf{A} \right\|}$ is the wrong instrument for large $t$ . The right one reads the eigenvalues, and the bridge to the discrete theory of §04 is the observation that the exponential turns the real part of an eigenvalue into a modulus.

Lemma 16.6.2 (The Spectrum of the Exponential)

Let $\mathbf{A} \in M_n(\mathbb{C})$ . Then $\operatorname{spec}(e^{\mathbf{A}}) = \{e^{\lambda} : \lambda \in \operatorname{spec}(\mathbf{A})\}$ , and $\rho(e^{\mathbf{A}}) = e^{\mu}, \qquad \mu = \max\{\operatorname{Re}\lambda : \lambda \in \operatorname{spec}(\mathbf{A})\} .$

Idea. The eigenvalues of $e^{\mathbf{A}}$ are the numbers $e^{\lambda}$ for $\lambda \in \operatorname{spec}(\mathbf{A})$ , which the Jordan form makes visible. Taking moduli turns $e^{\lambda}$ into $e^{\operatorname{Re}\lambda}$ , and the largest of finitely many such numbers is the one with the largest exponent.

Proof. By Theorem 10.3.1 there are an invertible $\mathbf{P}$ and a Jordan matrix $\mathbf{J}$ with $\mathbf{A} = \mathbf{P}\mathbf{J}\mathbf{P}^{-1}$ , and the diagonal of $\mathbf{J}$ lists the eigenvalues $\lambda_1, \dots, \lambda_n$ of $\mathbf{A}$ with multiplicity. By Theorem 10.9.4 (a), $e^{\mathbf{J}} = e^{\mathbf{P}^{-1}\mathbf{A}\mathbf{P}} = \mathbf{P}^{-1}e^{\mathbf{A}}\mathbf{P}$ . By Theorem 10.9.3 each block $e^{\mathbf{J}_k(\lambda)}$ is upper triangular with $e^{\lambda}$ on its diagonal, so $e^{\mathbf{J}}$ is upper triangular with diagonal $e^{\lambda_1}, \dots, e^{\lambda_n}$ ; hence $\operatorname{spec}(e^{\mathbf{J}}) = \{e^{\lambda_i}\}$ by Theorem 9.7.4 (b). Similar matrices have the same characteristic polynomial (Theorem 7.10.3), so $\operatorname{spec}(e^{\mathbf{A}}) = \operatorname{spec}(e^{\mathbf{J}})$ , which is the first claim.

For the second, $\lvert e^{\lambda}\rvert = e^{\operatorname{Re}\lambda}$ by fact (A1) of Chapter 10 §09, and $s \mapsto e^{s}$ is nondecreasing on $\mathbb{R}$ — for $h \ge 0$ , $e^{s+h} = e^{s}e^{h} \ge e^{s}$ by fact (A1) of Chapter 10 §09 — so the largest of the finitely many numbers $e^{\operatorname{Re}\lambda_i}$ is $e^{\mu}$ . This proves the lemma.

Chapter 12 §10 called $\mathbf{A}$ stable when every $\lambda \in \operatorname{spec}(\mathbf{A})$ has $\operatorname{Re}\lambda < 0$ (Definition 12.10.2). In the language of the lemma, $\mathbf{A}$ is stable exactly when $\rho(e^{\mathbf{A}}) < 1$ — the continuous-time condition and Chapter 10 §10’s discrete-time condition are the same condition, read through the exponential.

Theorem 16.6.3 (Exponential Decay of a Stable System)

Let $\mathbf{A} \in M_n(\mathbb{C})$ , let $\mu = \max\{\operatorname{Re}\lambda : \lambda \in \operatorname{spec}(\mathbf{A})\}$ , and let $\nu > \mu$ be real. Then there is a constant $C \ge 1$ , depending on $\mathbf{A}$ , on $\nu$ , and on the norm, with $\left\| e^{t\mathbf{A}} \right\| \le C\,e^{\nu t} \qquad \text{for every } t \ge 0 .$ In particular, if $\mathbf{A}$ is stable then $\nu$ may be taken negative, and every solution of $\mathbf{x}' = \mathbf{A}\mathbf{x}$ satisfies $\left\| \mathbf{x}(t) \right\| \le C e^{\nu t}\left\| \mathbf{x}(0) \right\| \xrightarrow[t \to \infty]{} 0 .$

Idea. Sample the flow at integer times and use §04. The matrix $\mathbf{M} = e^{-\nu}e^{\mathbf{A}}$ has $\rho(\mathbf{M}) = e^{\mu-\nu} < 1$ by Lemma 16.6.2, so Corollary 16.4.5 bounds all of its powers by one constant; unwinding the powers gives $\left\| e^{k\mathbf{A}} \right\| \le C_0e^{\nu k}$ at every integer $k$ . Between the integers, $e^{t\mathbf{A}} = e^{(t-k)\mathbf{A}}e^{k\mathbf{A}}$ with $0 \le t - k < 1$ , and the crude bound Theorem 16.6.1 (a) is more than enough to cover a gap of length $1$ . The whole point is that the crude bound is applied only on a bounded interval, where crudeness costs a constant and not a rate.

Proof. Put $\mathbf{M} = e^{-\nu}e^{\mathbf{A}}$ . Multiplying a matrix by a scalar $c$ multiplies every eigenvalue by $c$ , so by Lemma 16.6.2, $\rho(\mathbf{M}) = e^{-\nu}\rho(e^{\mathbf{A}}) = e^{-\nu}e^{\mu} = e^{\mu - \nu} < 1 ,$ using $\mu - \nu < 0$ . Fix any $q$ with $\rho(\mathbf{M}) < q < 1$ . By Corollary 16.4.5 (b) there is $C_0 \ge 1$ with $\left\| \mathbf{M}^{k} \right\| \le C_0q^{k} \le C_0$ for every integer $k \ge 0$ .

Since $\mathbf{A}$ commutes with itself, Theorem 10.9.4 (d) gives $e^{(s+u)\mathbf{A}} = e^{s\mathbf{A}}e^{u\mathbf{A}}$ for all real $s, u$ ; in particular $(e^{\mathbf{A}})^{k} = e^{k\mathbf{A}}$ by induction on $k$ . Hence $\mathbf{M}^{k} = e^{-\nu k}e^{k\mathbf{A}}$ , and $\left\| e^{k\mathbf{A}} \right\| = e^{\nu k}\left\| \mathbf{M}^{k} \right\| \le C_0\,e^{\nu k} \qquad (k = 0, 1, 2, \dots).$

Now let $t \ge 0$ and let $k$ be the integer part of $t$ , so that $k \ge 0$ and $0 \le t - k < 1$ . Then $e^{t\mathbf{A}} = e^{(t-k)\mathbf{A}}e^{k\mathbf{A}}$ , so by submultiplicativity and Theorem 16.6.1 (a), $\left\| e^{t\mathbf{A}} \right\| \le \left\| e^{(t-k)\mathbf{A}} \right\|\;\left\| e^{k\mathbf{A}} \right\| \le e^{(t-k)\left\| \mathbf{A} \right\|}\,C_0e^{\nu k} \le C_0\,e^{\left\| \mathbf{A} \right\|}\,e^{\nu k} ,$ using $0 \le t - k < 1$ in the last step. Finally $e^{\nu k} = e^{\nu t}e^{-\nu(t-k)} \le e^{\nu t}e^{\lvert\nu\rvert}$ , again because $0 \le t - k < 1$ . So the constant $C = C_0\,e^{\left\| \mathbf{A} \right\| + \lvert\nu\rvert} \ge 1$ works for every $t \ge 0$ .

For the last statement, suppose $\mathbf{A}$ is stable, so $\mu < 0$ , and choose $\nu \in (\mu, 0)$ . By Theorem 10.9.6 the unique solution with the given initial value is $\mathbf{x}(t) = e^{t\mathbf{A}}\mathbf{x}(0)$ , so $\left\| \mathbf{x}(t) \right\| \le \left\| e^{t\mathbf{A}} \right\|\left\| \mathbf{x}(0) \right\| \le Ce^{\nu t}\left\| \mathbf{x}(0) \right\|$ by Theorem 16.3.2 (a). Since $\nu < 0$ , $e^{\nu t} \to 0$ as $t \to \infty$ — this is fact (A5) of Chapter 10 §09 with $j = 0$ , quoted there and quoted again here — so $\left\| \mathbf{x}(t) \right\| \to 0$ , and by Corollary 16.2.4 the vectors $\mathbf{x}(t)$ tend to $\mathbf{0}$ . This proves the theorem.

Example (Decay after a detour)

Let $\mathbf{A} = \begin{pmatrix} -1 & 5 \\ 0 & -1 \end{pmatrix}$ . Find $\mu$ , compute $\left\| e^{t\mathbf{A}} \right\|_{\infty}$ exactly, and find the smallest constant $C$ that works in Theorem 16.6.3 for $\nu = -\tfrac12$ .

Solution. The eigenvalues. $\mathbf{A}$ is upper triangular with both diagonal entries $-1$ , so $\operatorname{spec}(\mathbf{A}) = \{-1\}$ (Theorem 9.7.4 (b)) and $\mu = -1 < 0$ : the matrix is stable.

The exponential. Write $\mathbf{A} = -\mathbf{I} + 5\mathbf{N}$ with $\mathbf{N} = \begin{psmallmatrix} 0&1\\mathbf{0}&0\end{psmallmatrix}$ . The two summands commute and $\mathbf{N}^{2} = \mathbf{O}$ , so Corollary 10.9.5 gives $e^{t\mathbf{A}} = e^{-t}(\mathbf{I} + 5t\mathbf{N}) = e^{-t}\begin{pmatrix} 1 & 5t \\ 0 & 1\end{pmatrix} .$ The absolute row sums are $e^{-t}(1 + 5t)$ and $e^{-t}$ , so $\left\| e^{t\mathbf{A}} \right\|_{\infty} = e^{-t}(1 + 5t)$ for $t \ge 0$ by Theorem 16.3.3 (b).

The transient. The function $t \mapsto e^{-t}(1+5t)$ has derivative $e^{-t}(4 - 5t)$ , so it rises from $1$ at $t = 0$ to $5e^{-4/5} \approx 2.247$ at $t = \tfrac45$ , and only then decays. A stable matrix does not decay from the start.

The constant. We need the smallest $C$ with $e^{-t}(1+5t) \le Ce^{-t/2}$ for all $t \ge 0$ , that is, $C = \max_{t \ge 0}(1+5t)e^{-t/2}$ . The derivative of $(1+5t)e^{-t/2}$ is $e^{-t/2}\bigl(5 - \tfrac12(1+5t)\bigr)$ , which is positive for $t < \tfrac95$ and negative for $t > \tfrac95$ , so by fact (A6) of the chapter introduction the maximum is at $t = \tfrac95$ and $C = 10\,e^{-9/10} \approx 4.066 .$ The rate $e^{-t/2}$ is honest and the constant pays for the hump.

Remark. What this settles from Chapter 12 §10, and what it does not. That section introduced the Lyapunov equation $\mathbf{A}\mathbf{X} + \mathbf{X}\mathbf{A}^{*} = -\mathbf{Q}$ and the word stable (Definition 12.10.2). It then stated, without proof, that a stable $\mathbf{A}$ and a positive definite $\mathbf{Q}$ produce a positive definite solution $\mathbf{X}$ , and conversely; it assigned the positivity half to Chapter 13, and assigned to this chapter “the consequence for differential equations — that a stable $\mathbf{A}$ makes every solution of $\dot{\mathbf{x}} = \mathbf{A}\mathbf{x}$ decay, with $\mathbf{x}^{*}\mathbf{X}\mathbf{x}$ as the quantity that measures the decay”.

Theorem 16.6.3 delivers the decay, with a rate, and does so without the Lyapunov equation at all. What it does not deliver is the existence of a positive definite $\mathbf{X}$ for a stable $\mathbf{A}$ ; that is Theorem 13.6.3, proved in Chapter 13 with no exponential anywhere in the argument, and nothing here depends on it. What can be said cheaply is the other half of the connection: any positive definite $\mathbf{X}$ satisfying $\mathbf{A}^{*}\mathbf{X} + \mathbf{X}\mathbf{A} = -\mathbf{Q}$ with $\mathbf{Q} \succ \mathbf{O}$ does measure the decay, in the sense that $\mathbf{x}^{*}\mathbf{X}\mathbf{x}$ is strictly decreasing along every nonzero solution of $\mathbf{x}' = \mathbf{A}\mathbf{x}$ . That is Exercise (C3: A quantity that measures the decay), and it is three lines once the product rule below is available. Note the transpose: the equation the decay computation wants is Definition 12.10.2 for the pair $(\mathbf{A}^{*}, \mathbf{Q})$ , and $\mathbf{A}$ is stable if and only if $\mathbf{A}^{*}$ is, since the eigenvalues of $\mathbf{A}^{*}$ are the conjugates of those of $\mathbf{A}$ .

16.6.3. Differentiating a matrix-valued function

A matrix whose entries vary — a family $\mathbf{A}(t)$ , a curve in $M_n(F)$ — is differentiated entrywise, exactly as Chapter 10 §09 differentiated $t \mapsto e^{t\mathbf{A}}$ . The two derivatives this book needs later, of $\det$ and of inversion, are algebraic facts with one analytic ingredient, and the ingredient is named first.

Definition 16.6.1 (Derivative of a Matrix-Valued Function)

Let $I \subseteq \mathbb{R}$ be an open interval and $\mathbf{A} \colon I \to M_{m \times n}(\mathbb{C})$ , written $\mathbf{A}(t) = (a_{ij}(t))$ . Call $\mathbf{A}$ differentiable at $t_0 \in I$ if every entry function $a_{ij} \colon I \to \mathbb{C}$ is differentiable at $t_0$ , and then set $\mathbf{A}'(t_0) \mathrel{:=} \bigl(a_{ij}'(t_0)\bigr) \in M_{m \times n}(\mathbb{C}) .$

The prime here always means the derivative in $t$ ; no dual map (Definition 5.4.1) appears anywhere in this section, so the notation cannot collide. By Corollary 16.2.4 the definition can be restated without coordinates: $\mathbf{A}$ is differentiable at $t_0$ with derivative $\mathbf{B}$ exactly when $\left\| \frac{\mathbf{A}(t) - \mathbf{A}(t_0)}{t - t_0} - \mathbf{B} \right\| \longrightarrow 0 \qquad (t \to t_0),$ in any norm at all. Both readings are used below, the entrywise one to import calculus and the norm one to take limits.

Remark. The calculus this section quotes. Let $f, g \colon I \to \mathbb{C}$ be differentiable at $t_0$ .

(D1) (Linearity and the product rule.) $f + g$ and $fg$ are differentiable at $t_0$ , with $(f+g)' = f' + g'$ and $(fg)' = f'g + fg'$ ; a constant function has derivative $0$ .
(D2) (Differentiable implies continuous.) $f$ is continuous at $t_0$ .

Neither is proved here. (D1) is Chapter 10 §09’s fact (A3), read for complex-valued functions of a real variable; (D2) is one more standard fact of the same kind, quoted here for the first time in this book. Both are of a piece with Chapter 14 §06’s Taylor assumption: standard one-variable calculus, quoted so that the theorems below have hypotheses to stand on. Everything that follows them is linear algebra.

Lemma 16.6.4 (Product Rule for Matrices)

Let $\mathbf{A} \colon I \to M_{m \times n}(\mathbb{C})$ and $\mathbf{B} \colon I \to M_{n \times p}(\mathbb{C})$ be differentiable at $t_0$ . Then so is $t \mapsto \mathbf{A}(t)\mathbf{B}(t)$ , with $(\mathbf{A}\mathbf{B})'(t_0) = \mathbf{A}'(t_0)\mathbf{B}(t_0) + \mathbf{A}(t_0)\mathbf{B}'(t_0) .$

Proof. Fix a position $(i, k)$ . The entry function is $t \mapsto \sum_{j=1}^{n}a_{ij}(t)b_{jk}(t)$ , a finite sum of products of functions differentiable at $t_0$ . By (D1) it is differentiable at $t_0$ with derivative $\sum_{j=1}^{n}\bigl(a_{ij}'(t_0)b_{jk}(t_0) + a_{ij}(t_0)b_{jk}'(t_0)\bigr),$ which is the $(i,k)$ -entry of $\mathbf{A}'(t_0)\mathbf{B}(t_0) + \mathbf{A}(t_0)\mathbf{B}'(t_0)$ . Since this holds at every position, Definition 16.6.1 gives the claim. Note that the order of the factors is preserved throughout: matrices do not commute, and the product rule must not be written $2\mathbf{A}\mathbf{A}'$ even for $\mathbf{B} = \mathbf{A}$ . This proves the lemma.

16.6.4. The derivative of the inverse

Nothing so far says that $t \mapsto \mathbf{A}(t)^{-1}$ is even defined near $t_0$ , let alone differentiable. Both come from §05.

Theorem 16.6.5 (Derivative of the Inverse)

Let $\mathbf{A} \colon I \to M_n(\mathbb{C})$ be differentiable at $t_0 \in I$ , with $\mathbf{A}(t_0)$ invertible. Then $\mathbf{A}(t)$ is invertible for all $t$ in some open interval around $t_0$ , the function $t \mapsto \mathbf{A}(t)^{-1}$ is differentiable at $t_0$ , and $\bigl(\mathbf{A}^{-1}\bigr)'(t_0) = -\,\mathbf{A}(t_0)^{-1}\,\mathbf{A}'(t_0)\,\mathbf{A}(t_0)^{-1} .$

Idea. The formula is what one gets by differentiating $\mathbf{A}\mathbf{A}^{-1} = \mathbf{I}$ with Lemma 16.6.4 and solving — but that argument presupposes what has to be proved, namely that $\mathbf{A}^{-1}$ is differentiable. So run it in the honest order. The difference of two inverses factors as $\mathbf{X}^{-1} - \mathbf{Y}^{-1} = \mathbf{X}^{-1}(\mathbf{Y} - \mathbf{X})\mathbf{Y}^{-1}$ ; divide by $t - t_0$ and the middle factor becomes a difference quotient, which converges by hypothesis, while the outer factor converges because inversion is continuous (Corollary 16.5.2). The limit of the product is the product of the limits, and that is the formula.

Proof. Write $\mathbf{A}_0 = \mathbf{A}(t_0)$ . By (D2) every entry of $\mathbf{A}$ is continuous at $t_0$ , so $\mathbf{A}(t) \to \mathbf{A}_0$ entrywise as $t \to t_0$ , hence $\left\| \mathbf{A}(t) - \mathbf{A}_0 \right\| \to 0$ by Corollary 16.2.4. By Corollary 16.5.2 there is therefore an open interval $J \ni t_0$ on which $\left\| \mathbf{A}(t) - \mathbf{A}_0 \right\| < 1/\left\| \mathbf{A}_0^{-1} \right\|$ , so that $\mathbf{A}(t)$ is invertible for $t \in J$ , and on which $\left\| \mathbf{A}(t)^{-1} - \mathbf{A}_0^{-1} \right\| \to 0$ as $t \to t_0$ .

Let $t \in J$ with $t \ne t_0$ . Multiplying $\mathbf{A}_0 - \mathbf{A}(t)$ on the left by $\mathbf{A}(t)^{-1}$ and on the right by $\mathbf{A}_0^{-1}$ , $\mathbf{A}(t)^{-1} - \mathbf{A}_0^{-1} = -\,\mathbf{A}(t)^{-1}\bigl(\mathbf{A}(t) - \mathbf{A}_0\bigr)\mathbf{A}_0^{-1} .$ Divide by $t - t_0$ : $\frac{\mathbf{A}(t)^{-1} - \mathbf{A}_0^{-1}}{t - t_0} = -\,\mathbf{A}(t)^{-1}\,\frac{\mathbf{A}(t) - \mathbf{A}_0}{t - t_0}\,\mathbf{A}_0^{-1} .$ As $t \to t_0$ the middle factor tends to $\mathbf{A}'(t_0)$ , by Definition 16.6.1. The left factor tends to $\mathbf{A}_0^{-1}$ , by the first paragraph. Products of convergent matrix families converge to the product of the limits: if $\left\| \mathbf{X}(t) - \mathbf{X} \right\| \to 0$ and $\left\| \mathbf{Y}(t) - \mathbf{Y} \right\| \to 0$ then $\left\| \mathbf{X}(t)\mathbf{Y}(t) - \mathbf{X}\mathbf{Y} \right\| \le \left\| \mathbf{X}(t) - \mathbf{X} \right\|\left\| \mathbf{Y}(t) \right\| + \left\| \mathbf{X} \right\|\left\| \mathbf{Y}(t) - \mathbf{Y} \right\| ,$ and $\left\| \mathbf{Y}(t) \right\|$ is bounded near $t_0$ because $\left\| \mathbf{Y}(t) \right\| \le \left\| \mathbf{Y} \right\| + \left\| \mathbf{Y}(t) - \mathbf{Y} \right\|$ . Applying this twice, the right-hand side converges to $-\mathbf{A}_0^{-1}\mathbf{A}'(t_0)\mathbf{A}_0^{-1}$ . Hence the difference quotient on the left converges, which by Definition 16.6.1 is exactly the differentiability of $t \mapsto \mathbf{A}(t)^{-1}$ at $t_0$ , with the stated derivative. This proves the theorem.

Warning. The two copies of $\mathbf{A}^{-1}$ sit on either side and cannot be combined. The scalar formula $(1/a)' = -a'/a^{2}$ tempts one to write $-\mathbf{A}^{-2}\mathbf{A}'$ , and that is wrong unless $\mathbf{A}'(t_0)$ commutes with $\mathbf{A}(t_0)$ . For $\mathbf{A}(t) = \begin{psmallmatrix} 1 & t \\ 0 & 2\end{psmallmatrix}$ at $t = 0$ : we have $\mathbf{A}(0)^{-1} = \operatorname{diag}(1, \tfrac12)$ and $\mathbf{A}'(0) = \begin{psmallmatrix} 0&1\\mathbf{0}&0\end{psmallmatrix}$ , so the correct answer is $-\operatorname{diag}(1,\tfrac12)\begin{psmallmatrix} 0&1\\mathbf{0}&0\end{psmallmatrix}\operatorname{diag}(1,\tfrac12) = \begin{psmallmatrix} 0 & -1/2 \\ 0 & 0\end{psmallmatrix}$ , whereas $-\mathbf{A}(0)^{-2}\mathbf{A}'(0) = -\begin{psmallmatrix} 1&0\\mathbf{0}&1/4\end{psmallmatrix}\begin{psmallmatrix} 0&1\\mathbf{0}&0\end{psmallmatrix} = \begin{psmallmatrix} 0 & -1 \\ 0 & 0\end{psmallmatrix}$ .

16.6.5. Jacobi’s formula

The determinant is a polynomial in the entries, so differentiating it needs no analysis beyond (D1); the work is in recognizing the answer.

Theorem 16.6.6 (Jacobi’s Formula)

Let $\mathbf{A} \colon I \to M_n(\mathbb{C})$ be differentiable at $t_0 \in I$ . Then $t \mapsto \det\mathbf{A}(t)$ is differentiable at $t_0$ , with $\frac{d}{dt}\Big|_{t_0}\det\mathbf{A}(t) = \operatorname{tr}\bigl(\operatorname{adj}\mathbf{A}(t_0)\,\mathbf{A}'(t_0)\bigr) .$ If moreover $\mathbf{A}(t_0)$ is invertible, this equals $\det\mathbf{A}(t_0)\,\operatorname{tr}\bigl(\mathbf{A}(t_0)^{-1}\mathbf{A}'(t_0)\bigr)$ .

Idea. Use the Leibniz formula, which writes $\det$ as a sum of $n!$ products of $n$ entries, one from each column. The product rule differentiates each such product into $n$ terms, the $j$ -th of which has column $j$ differentiated and the rest untouched. Collecting by $j$ gives $n$ determinants, the $j$ -th being $\mathbf{A}$ with its $j$ -th column replaced by $\mathbf{a}_j'$ . Expanding that determinant along column $j$ (Theorem 7.6.1) produces $\sum_i a_{ij}'C_{ij}$ , and summing over $j$ gives a double sum which is precisely a trace against the adjugate.

Proof. For $n = 1$ the statement reads $\frac{d}{dt}\big|_{t_0}a_{11}(t) = a_{11}'(t_0)$ , since $\operatorname{adj}\mathbf{A} = (1)$ by Definition 7.6.2, so assume $n \ge 2$ ; that is what lets us speak of cofactors. Write $\mathbf{A}(t) = (a_{ij}(t))$ with columns $\mathbf{a}_1(t), \dots, \mathbf{a}_n(t)$ , and write $C_{ij}$ for the cofactors of $\mathbf{A}(t_0)$ (Definition 7.6.1). By Theorem 7.4.3, $\det\mathbf{A}(t) = \sum_{\sigma \in S_n}\operatorname{sgn}(\sigma)\,a_{\sigma(1)1}(t)\cdots a_{\sigma(n)n}(t) ,$ a finite sum of products of $n$ functions each differentiable at $t_0$ . By (D1), applied $n - 1$ times to each product and then to the sum, $\det\mathbf{A}$ is differentiable at $t_0$ with $\frac{d}{dt}\Big|_{t_0}\det\mathbf{A} = \sum_{j=1}^{n}\ \sum_{\sigma \in S_n}\operatorname{sgn}(\sigma)\,a_{\sigma(j)j}'(t_0)\prod_{l \ne j}a_{\sigma(l)l}(t_0) ,$ where the outer sum collects, for each $j$ , the terms in which the $j$ -th factor was the one differentiated. By Theorem 7.4.3 read backwards, the inner sum is the determinant of the matrix $\mathbf{A}_j$ obtained from $\mathbf{A}(t_0)$ by replacing its $j$ -th column with $\mathbf{a}_j'(t_0)$ . Expanding $\det\mathbf{A}_j$ along that column with Theorem 7.6.1 (a), and using that the cofactors of column $j$ do not involve column $j$ and are therefore the cofactors $C_{ij}$ of $\mathbf{A}(t_0)$ itself, $\det \mathbf{A}_j = \sum_{i=1}^{n}a_{ij}'(t_0)\,C_{ij} .$ Summing over $j$ , $\frac{d}{dt}\Big|_{t_0}\det\mathbf{A} = \sum_{j=1}^{n}\sum_{i=1}^{n}C_{ij}\,a_{ij}'(t_0) .$ It remains to recognize the right-hand side. By Definition 7.6.2, $(\operatorname{adj}\mathbf{A}(t_0))_{ji} = C_{ij}$ , so $\operatorname{tr}\bigl(\operatorname{adj}\mathbf{A}(t_0)\,\mathbf{A}'(t_0)\bigr) = \sum_{j=1}^{n}\sum_{i=1}^{n}(\operatorname{adj}\mathbf{A}(t_0))_{ji}\,a_{ij}'(t_0) ,$ which is the same double sum. For the last claim, if $\mathbf{A}(t_0)$ is invertible then $\operatorname{adj}\mathbf{A}(t_0) = \det\mathbf{A}(t_0)\,\mathbf{A}(t_0)^{-1}$ by Theorem 7.6.2, and the trace is linear (Theorem 0.9.6), so the scalar $\det\mathbf{A}(t_0)$ comes out in front. This proves the theorem.

Corollary 16.6.7 (The Determinant’s Derivative at the Identity Is the Trace)

For every $\mathbf{B} \in M_n(\mathbb{C})$ , $\frac{d}{dt}\Big|_{t=0}\det(\mathbf{I} + t\mathbf{B}) = \operatorname{tr}\mathbf{B} .$

Proof. Apply Theorem 16.6.6 to $\mathbf{A}(t) = \mathbf{I} + t\mathbf{B}$ , which is differentiable with $\mathbf{A}'(t) = \mathbf{B}$ and $\mathbf{A}(0) = \mathbf{I}$ . Since $\mathbf{I}$ is invertible with $\mathbf{I}^{-1} = \mathbf{I}$ and $\det\mathbf{I} = 1$ , the second form of the formula gives $\operatorname{tr}(\mathbf{I}\mathbf{B}) = \operatorname{tr}\mathbf{B}$ .

So the determinant, which is a product of $n$ numbers, has the trace, a sum of $n$ numbers, as its linearization at $\mathbf{I}$ . That is the honest source of the identity $\det e^{\mathbf{A}} = e^{\operatorname{tr}\mathbf{A}}$ of Theorem 10.9.4 (e), and Exercise (C1: The determinant of the exponential, again) reproves that identity along these lines.

Example (Both derivatives on one family)

Let $\mathbf{A}(t) = \begin{pmatrix} 1 + t & t^{2} \\ t & 1 \end{pmatrix}$ .

(a) Compute $\frac{d}{dt}\det\mathbf{A}(t)$ directly and by Theorem 16.6.6.
(b) Compute $(\mathbf{A}^{-1})'(0)$ by Theorem 16.6.5, and check it against a direct expansion.

Solution.

(a) Directly. $\det\mathbf{A}(t) = (1+t)\cdot 1 - t^{2}\cdot t = 1 + t - t^{3}$ , so the derivative is $1 - 3t^{2}$ .
By the formula. $\operatorname{adj}\mathbf{A}(t) = \begin{psmallmatrix} 1 & -t^{2} \\ -t & 1+t \end{psmallmatrix}$ (Definition 7.6.2) and $\mathbf{A}'(t) = \begin{psmallmatrix} 1 & 2t \\ 1 & 0\end{psmallmatrix}$ , so $\operatorname{adj}\mathbf{A}(t)\,\mathbf{A}'(t) = \begin{pmatrix} 1 - t^{2} & 2t \\ 1 & -2t^{2} \end{pmatrix},$ whose trace is $1 - t^{2} - 2t^{2} = 1 - 3t^{2}$ . The two agree.
(b) $\mathbf{A}(0) = \mathbf{I}$ and $\mathbf{A}'(0) = \begin{psmallmatrix} 1 & 0 \\ 1 & 0\end{psmallmatrix}$ , so Theorem 16.6.5 gives $(\mathbf{A}^{-1})'(0) = -\mathbf{I}\begin{pmatrix} 1 & 0 \\ 1 & 0\end{pmatrix}\mathbf{I} = \begin{pmatrix} -1 & 0 \\ -1 & 0\end{pmatrix} .$

Check. $\mathbf{A}(t)^{-1} = \dfrac{1}{1 + t - t^{3}}\begin{psmallmatrix} 1 & -t^{2} \\ -t & 1+t\end{psmallmatrix}$ . Near $t = 0$ the scalar factor is $1 - t + (\text{terms in } t^{2})$ , so $\mathbf{A}(t)^{-1} = \begin{pmatrix} 1 - t & 0 \\ -t & 1 \end{pmatrix} + (\text{terms in } t^{2}),$ whose derivative at $0$ is $\begin{psmallmatrix} -1 & 0 \\ -1 & 0\end{psmallmatrix}$ , as predicted. The formula did the work that the quotient rule would otherwise have had to do four times.

Quick check. Theorem 16.6.6 holds with no invertibility hypothesis, yet the second form of the formula has one. Where exactly is invertibility used, and what does the formula say at a $t_0$ where $\det\mathbf{A}(t_0) = 0$ ?

Solution. Only in the last line, where $\operatorname{adj}\mathbf{A}(t_0)$ is rewritten as $\det\mathbf{A}(t_0)\mathbf{A}(t_0)^{-1}$ . The adjugate itself is defined for every square matrix (Definition 7.6.2), so the first form survives. At a $t_0$ with $\det\mathbf{A}(t_0) = 0$ it still gives a number: for $\mathbf{A}(t) = \operatorname{diag}(t, t)$ at $t_0 = 0$ we get $\operatorname{adj}\mathbf{A}(0) = \mathbf{O}$ , so the derivative of $\det\mathbf{A}(t) = t^{2}$ is $0$ there, which is correct. The second form would read “ $0$ times something undefined”.

16.6.6. Exercises

A. Check your understanding

Exercise (A1)

(a) State the three bounds of Theorem 16.6.1, and say which property of the norm each one needs.
(b) Determine whether the following statement is correct, and justify your answer: if $\left\| \mathbf{A} \right\|_2$ is large then $\left\| e^{\mathbf{A}} \right\|_2$ is large.
(c) State Theorem 16.6.3, and say what the constant $C$ is paying for.
(d) State Jacobi’s formula (Theorem 16.6.6), and name the fact from calculus its proof quotes.
(e) Write down the derivative of $t \mapsto \mathbf{A}(t)^{-1}$ and explain in one sentence why it is not $-\mathbf{A}^{-2}\mathbf{A}'$ .

Solution.

(a) $\left\| e^{\mathbf{A}} \right\| \le e^{\left\| \mathbf{A} \right\|}$ , $\left\| e^{\mathbf{A}} - \mathbf{I} \right\| \le e^{\left\| \mathbf{A} \right\|} - 1$ , and $\left\| e^{\mathbf{A}} - \mathbf{I} - \mathbf{A} \right\| \le e^{\left\| \mathbf{A} \right\|} - 1 - \left\| \mathbf{A} \right\| \le \left\| \mathbf{A} \right\|^{2}e^{\left\| \mathbf{A} \right\|}/2$ . All three need subadditivity (to bound a sum term by term) and submultiplicativity (to get $\left\| \mathbf{A}^{m} \right\| \le \left\| \mathbf{A} \right\|^{m}$ ); the first also needs $\left\| \mathbf{I} \right\| = 1$ , which is what makes the $m = 0$ term equal $1$ .
(b) Incorrect. For $\mathbf{A} = t\begin{psmallmatrix} 0 & 1 \\ -1 & 0\end{psmallmatrix}$ we have $\left\| \mathbf{A} \right\|_2 = t$ , arbitrarily large, while $e^{\mathbf{A}}$ is a rotation matrix and $\left\| e^{\mathbf{A}} \right\|_2 = 1$ (Example (Where the bound is exact, and where it is useless)).
(c) If $\mu$ is the largest real part of an eigenvalue of $\mathbf{A}$ and $\nu > \mu$ , there is $C \ge 1$ with $\left\| e^{t\mathbf{A}} \right\| \le Ce^{\nu t}$ for all $t \ge 0$ . The constant pays for the transient: $\left\| e^{t\mathbf{A}} \right\|$ may first grow, as in Example (Decay after a detour), and it also pays for the gap between $\nu$ and $\mu$ .
(d) $\frac{d}{dt}\det\mathbf{A}(t) = \operatorname{tr}(\operatorname{adj}\mathbf{A}(t)\,\mathbf{A}'(t))$ , for a matrix-valued $\mathbf{A}$ differentiable at $t$ . The quoted fact is (D1): the linearity of differentiation and the product rule for complex-valued functions of a real variable.
(e) $(\mathbf{A}^{-1})' = -\mathbf{A}^{-1}\mathbf{A}'\mathbf{A}^{-1}$ . The two copies of $\mathbf{A}^{-1}$ sit on opposite sides of $\mathbf{A}'$ , and matrices do not commute, so they may not be collected into $\mathbf{A}^{-2}$ unless $\mathbf{A}'$ commutes with $\mathbf{A}$ .

B. Practice

Exercise (B1: Bound against truth)

For each matrix, compute $\left\| e^{\mathbf{A}} \right\|_{\infty}$ exactly and compare with $e^{\left\| \mathbf{A} \right\|_{\infty}}$ .

(a) $\mathbf{A} = \begin{pmatrix} 0 & 3 \\ 0 & 0 \end{pmatrix}$ .
(b) $\mathbf{A} = \operatorname{diag}(1, -4)$ .

Solution.

(a) $\mathbf{A}^{2} = \mathbf{O}$ , so the series stops: $e^{\mathbf{A}} = \mathbf{I} + \mathbf{A} = \begin{psmallmatrix} 1 & 3 \\ 0 & 1\end{psmallmatrix}$ , with $\left\| e^{\mathbf{A}} \right\|_{\infty} = 4$ . Also $\left\| \mathbf{A} \right\|_{\infty} = 3$ , so the bound is $e^{3} \approx 20.09$ . Correct, and loose by a factor of about $5$ .
(b) $e^{\mathbf{A}} = \operatorname{diag}(e, e^{-4})$ , so $\left\| e^{\mathbf{A}} \right\|_{\infty} = e \approx 2.718$ . Here $\left\| \mathbf{A} \right\|_{\infty} = 4$ and the bound is $e^{4} \approx 54.6$ . The bound is governed by the entry of largest modulus, $-4$ , while the exponential is governed by the largest real part, $1$ . This is the discrepancy Theorem 16.6.3 exists to remove.

Exercise (B2: A rate for a stable matrix)

Let $\mathbf{A} = \begin{pmatrix} -2 & 4 \\ 0 & -2 \end{pmatrix}$ .

(a) Compute $\mu$ and $e^{t\mathbf{A}}$ , and give $\left\| e^{t\mathbf{A}} \right\|_{\infty}$ for $t \ge 0$ .
(b) Find the largest value of $\left\| e^{t\mathbf{A}} \right\|_{\infty}$ on $t \ge 0$ .
(c) Find the smallest $C$ that works in Theorem 16.6.3 for $\nu = -1$ .

Solution.

(a) $\mathbf{A}$ is upper triangular with diagonal $-2, -2$ , so $\operatorname{spec}(\mathbf{A}) = \{-2\}$ (Theorem 9.7.4 (b)) and $\mu = -2$ . Writing $\mathbf{A} = -2\mathbf{I} + 4\mathbf{N}$ with $\mathbf{N} = \begin{psmallmatrix} 0&1\\mathbf{0}&0\end{psmallmatrix}$ , the summands commute and $\mathbf{N}^2 = \mathbf{O}$ , so by Corollary 10.9.5 $e^{t\mathbf{A}} = e^{-2t}\begin{pmatrix} 1 & 4t \\ 0 & 1 \end{pmatrix}, \qquad \left\| e^{t\mathbf{A}} \right\|_{\infty} = e^{-2t}(1 + 4t) .$
(b) The derivative of $e^{-2t}(1+4t)$ is $e^{-2t}(4 - 2 - 8t) = e^{-2t}(2 - 8t)$ , which is positive for $t < \tfrac14$ and negative after, so by fact (A6) the maximum is at $t = \tfrac14$ . The value there is $2e^{-1/2} \approx 1.213$ .
(c) We need the smallest $C$ with $e^{-2t}(1+4t) \le Ce^{-t}$ , that is, $C = \max_{t \ge 0}(1+4t)e^{-t}$ . Its derivative is $e^{-t}(4 - 1 - 4t) = e^{-t}(3 - 4t)$ , positive for $t < \tfrac34$ and negative after, so by fact (A6) the maximum is at $t = \tfrac34$ and $C = 4e^{-3/4} \approx 1.890$ .

Exercise (B3: Two derivatives)

Let $\mathbf{A}(t) = \begin{pmatrix} \cos t & -\sin t \\ \sin t & \cos t \end{pmatrix}$ .

(a) Compute $\frac{d}{dt}\det\mathbf{A}(t)$ by Theorem 16.6.6 and check it against the exact value of $\det\mathbf{A}(t)$ .
(b) Compute $(\mathbf{A}^{-1})'(t)$ by Theorem 16.6.5 and check it by differentiating $\mathbf{A}(t)^{-1}$ directly.

Solution.

(a) $\det\mathbf{A}(t) = \cos^{2}t + \sin^{2}t = 1$ , so the derivative is $0$ . By the formula: $\operatorname{adj}\mathbf{A}(t) = \begin{psmallmatrix} \cos t & \sin t \\ -\sin t & \cos t \end{psmallmatrix}$ and $\mathbf{A}'(t) = \begin{psmallmatrix} -\sin t & -\cos t \\ \cos t & -\sin t\end{psmallmatrix}$ , so $\operatorname{adj}\mathbf{A}(t)\mathbf{A}'(t) = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix},$ whose trace is $0$ . The two agree. (The diagonal entries are $-\cos t\sin t + \sin t\cos t$ and $\sin t\cos t - \cos t\sin t$ , both zero.)
(b) Since $\det\mathbf{A}(t) = 1$ , $\mathbf{A}(t)^{-1} = \operatorname{adj}\mathbf{A}(t) = \begin{psmallmatrix} \cos t & \sin t \\ -\sin t & \cos t\end{psmallmatrix}$ , whose entrywise derivative is $\begin{psmallmatrix} -\sin t & \cos t \\ -\cos t & -\sin t\end{psmallmatrix}$ . By the formula, using the product $\operatorname{adj}\mathbf{A}(t)\mathbf{A}'(t)$ computed in (a), $-\mathbf{A}^{-1}\mathbf{A}'\mathbf{A}^{-1} = -\begin{pmatrix} 0 & -1 \\ 1 & 0\end{pmatrix}\begin{pmatrix} \cos t & \sin t \\ -\sin t & \cos t\end{pmatrix} = \begin{pmatrix} -\sin t & \cos t \\ -\cos t & -\sin t \end{pmatrix} .$ The two agree.

C. Going deeper

Exercise (C1: The determinant of the exponential, again)

Let $\mathbf{A} \in M_n(\mathbb{C})$ and put $\varphi(t) = \det e^{t\mathbf{A}}$ .

(a) Prove that $\varphi$ is differentiable with $\varphi'(t) = (\operatorname{tr}\mathbf{A})\varphi(t)$ .
(b) Hence deduce $\det e^{\mathbf{A}} = e^{\operatorname{tr}\mathbf{A}}$ , recovering Theorem 10.9.4 (e).

Hint for (b): consider $\psi(t) = e^{-t\operatorname{tr}\mathbf{A}}\varphi(t)$ .

Solution.

(a) The function $t \mapsto e^{t\mathbf{A}}$ is differentiable with derivative $\mathbf{A} e^{t\mathbf{A}}$ (Theorem 10.9.4 (b)), and $e^{t\mathbf{A}}$ is invertible with inverse $e^{-t\mathbf{A}}$ (Theorem 10.9.4 (c)). So Theorem 16.6.6 applies in its second form: $\varphi'(t) = \det(e^{t\mathbf{A}})\,\operatorname{tr}\bigl(e^{-t\mathbf{A}}\mathbf{A} e^{t\mathbf{A}}\bigr) .$ By Theorem 0.9.6 (3) with the two factors $e^{-t\mathbf{A}}$ and $\mathbf{A} e^{t\mathbf{A}}$ , we get $\operatorname{tr}(e^{-t\mathbf{A}}\mathbf{A} e^{t\mathbf{A}}) = \operatorname{tr}(\mathbf{A} e^{t\mathbf{A}}e^{-t\mathbf{A}}) = \operatorname{tr}\mathbf{A}$ , giving $\varphi'(t) = (\operatorname{tr}\mathbf{A})\varphi(t)$ .
(b) Put $c = \operatorname{tr}\mathbf{A}$ and $\psi(t) = e^{-ct}\varphi(t)$ . By the product rule (D1) and (a), $\psi'(t) = -c\,e^{-ct}\varphi(t) + e^{-ct}c\,\varphi(t) = 0$ for every $t$ , where $\frac{d}{dt}e^{-ct} = -ce^{-ct}$ is Theorem 10.9.4 (b) in the case $n = 1$ . A differentiable function with vanishing derivative is constant — fact (A4) of Chapter 10 §09 — so $\psi(t) = \psi(0) = \det(e^{0}) = \det\mathbf{I} = 1$ . Hence $\varphi(t) = e^{ct}$ , and $t = 1$ gives $\det e^{\mathbf{A}} = e^{\operatorname{tr}\mathbf{A}}$ .

Exercise (C2: Comparing two exponentials)

Let $\mathbf{A}, \mathbf{B} \in M_n(\mathbb{C})$ with $\mathbf{A}\mathbf{B} = \mathbf{B}\mathbf{A}$ .

(a) Prove that $\left\| e^{\mathbf{A}} - e^{\mathbf{B}} \right\| \le e^{\left\| \mathbf{B} \right\|}\bigl(e^{\left\| \mathbf{A} - \mathbf{B} \right\|} - 1\bigr)$ .
(b) Deduce that $\mathbf{M} \mapsto e^{\mathbf{M}}$ is continuous at every $\mathbf{M}$ that commutes with a neighborhood of itself, and identify those $\mathbf{M}$ : they are exactly the scalar matrices $c\mathbf{I}$ .
(c) Say exactly where the commuting hypothesis is used, and why the argument cannot simply be dropped.

Solution.

(a) Since $\mathbf{A}$ and $\mathbf{B}$ commute, so do $\mathbf{B}$ and $\mathbf{A} - \mathbf{B}$ . By Theorem 10.9.4 (d), $e^{\mathbf{A}} = e^{\mathbf{B} + (\mathbf{A} - \mathbf{B})} = e^{\mathbf{B}}e^{\mathbf{A} - \mathbf{B}}$ . Hence $e^{\mathbf{A}} - e^{\mathbf{B}} = e^{\mathbf{B}}\bigl(e^{\mathbf{A}-\mathbf{B}} - \mathbf{I}\bigr) ,$ and submultiplicativity with Theorem 16.6.1 (a) and (b) gives $\left\| e^{\mathbf{A}} - e^{\mathbf{B}} \right\| \le \left\| e^{\mathbf{B}} \right\|\,\left\| e^{\mathbf{A}-\mathbf{B}} - \mathbf{I} \right\| \le e^{\left\| \mathbf{B} \right\|}\bigl(e^{\left\| \mathbf{A}-\mathbf{B} \right\|} - 1\bigr) .$
(b) Take $\mathbf{B} = c\mathbf{I}$ , which commutes with every matrix. For $\mathbf{A}$ with $\left\| \mathbf{A} - c\mathbf{I} \right\| = \delta$ , part (a) gives $\left\| e^{\mathbf{A}} - e^{c}\mathbf{I} \right\| \le e^{\lvert c\rvert}(e^{\delta} - 1)$ , and $e^{\delta} - 1 \to 0$ as $\delta \to 0$ , because $t \mapsto e^{t}$ is differentiable by fact (A2) of Chapter 10 §09 and so continuous at $0$ . So $e^{\mathbf{A}} \to e^{c\mathbf{I}}$ as $\mathbf{A} \to c\mathbf{I}$ .
(c) Only in the first line, where $e^{\mathbf{B} + (\mathbf{A}-\mathbf{B})}$ is split as $e^{\mathbf{B}}e^{\mathbf{A}-\mathbf{B}}$ ; Theorem 10.9.4 (d) has commutation as a hypothesis, and Chapter 10 §09 exhibits matrices with $e^{\mathbf{A}+\mathbf{B}} \ne e^{\mathbf{A}}e^{\mathbf{B}}$ . Without it there is no factorization to take norms of, and a different argument is needed. (The inequality itself is in fact true in general, but proving it needs an integral representation of $e^{\mathbf{A}} - e^{\mathbf{B}}$ , which this book does not have.)

Exercise (C3: A quantity that measures the decay)

Let $\mathbf{A}, \mathbf{X}, \mathbf{Q} \in M_n(\mathbb{C})$ with $\mathbf{X}$ and $\mathbf{Q}$ positive definite (Definition 13.1.1) and $\mathbf{A}^{*}\mathbf{X} + \mathbf{X}\mathbf{A} = -\mathbf{Q} .$ Let $\mathbf{x} \colon \mathbb{R} \to \mathbb{C}^{n}$ be differentiable with $\mathbf{x}' = \mathbf{A}\mathbf{x}$ , and put $V(t) = \mathbf{x}(t)^{*}\mathbf{X}\mathbf{x}(t)$ .

(a) Prove that $V$ is real-valued and differentiable, with $V'(t) = \mathbf{x}(t)^{*}(\mathbf{A}^{*}\mathbf{X} + \mathbf{X}\mathbf{A})\mathbf{x}(t)$ .
(b) Deduce that if $\mathbf{x}(0) \ne \mathbf{0}$ , then $V$ is strictly decreasing.
(c) Explain how this relates to the Lyapunov equation of Definition 12.10.2, whose unknown sits between $\mathbf{A}$ and $\mathbf{A}^{*}$ the other way round.

Hint for (a): $\mathbf{x}^{*}$ is a $1 \times n$ matrix-valued function, and $\overline{f}' = \overline{f'}$ for a differentiable $f \colon \mathbb{R} \to \mathbb{C}$ .

Solution.

(a) $V(t)$ is a $1 \times 1$ matrix, and $V(t)^{*} = \mathbf{x}^{*}\mathbf{X}^{*}\mathbf{x} = \mathbf{x}^{*}\mathbf{X}\mathbf{x} = V(t)$ since $\mathbf{X}$ is Hermitian, so $V(t)$ is real. Each entry of $t \mapsto \mathbf{x}(t)^{*}$ is the conjugate of an entry of $\mathbf{x}$ , hence differentiable with $(\mathbf{x}^{*})' = (\mathbf{x}')^{*} = (\mathbf{A}\mathbf{x})^{*} = \mathbf{x}^{*}\mathbf{A}^{*}$ . By Lemma 16.6.4, applied twice to the product $\mathbf{x}^{*}\cdot\mathbf{X}\cdot\mathbf{x}$ and using $\mathbf{X}' = \mathbf{O}$ , $V'(t) = \mathbf{x}^{*}\mathbf{A}^{*}\mathbf{X}\mathbf{x} + \mathbf{x}^{*}\mathbf{X}\mathbf{A}\mathbf{x} = \mathbf{x}^{*}(\mathbf{A}^{*}\mathbf{X} + \mathbf{X}\mathbf{A})\mathbf{x} .$
(b) By hypothesis $V'(t) = -\mathbf{x}(t)^{*}\mathbf{Q}\mathbf{x}(t)$ . If $\mathbf{x}(t_1) = \mathbf{0}$ for some $t_1$ , then by Theorem 10.9.6 $\mathbf{x}(t) = e^{t\mathbf{A}}\mathbf{x}(0)$ for every $t$ , and $e^{t_1\mathbf{A}}$ is invertible by Theorem 10.9.4 (c), so $\mathbf{x}(0) = \mathbf{0}$ , contradicting the hypothesis. So $\mathbf{x}(t) \ne \mathbf{0}$ for every $t$ , and $\mathbf{Q} \succ \mathbf{O}$ gives $\mathbf{x}(t)^{*}\mathbf{Q}\mathbf{x}(t) > 0$ , hence $V'(t) < 0$ for every $t$ . A differentiable real function with everywhere negative derivative is strictly decreasing; that last step is the mean value theorem, one more fact of one-variable calculus quoted here and not proved.
(c) Definition 12.10.2 writes the equation as $\mathbf{A}\mathbf{X} + \mathbf{X}\mathbf{A}^{*} = -\mathbf{Q}$ ; the equation used here is that one for the pair $(\mathbf{A}^{*}, \mathbf{Q})$ . The two are interchanged by replacing $\mathbf{A}$ with $\mathbf{A}^{*}$ , which changes nothing about stability, since the eigenvalues of $\mathbf{A}^{*}$ are the complex conjugates of those of $\mathbf{A}$ and conjugation preserves real parts. So the decay of $V$ is exactly the statement Chapter 12 §10 pointed forward to, for whichever of the two conventions one adopts, and such an $\mathbf{X}$ does exist: Theorem 13.6.3 applied to $\mathbf{A}^{*}$ produces one.