17.11  Which Functions Respect the Order

Chapter 13 §05 ended on two facts that sit awkwardly together. If $\mathbf{A}⪰\mathbf{B}⪰\mathbf{O}$, then ${\mathbf{A}}^{1/2}⪰{\mathbf{B}}^{1/2}$ (Proposition 13.5.4); but ${\mathbf{A}}^2⪰{\mathbf{B}}^2$ can fail (Example (Squaring does not preserve the Loewner order)). For numbers, both $t\mapsto t^{1/2}$ and $t\mapsto t^2$ are increasing on $[0,\mathrm{\infty })$, and both preserve inequalities. This section names the property that separates them and proves it for every function the book can reach. It then compares that property with what §02’s min–max theorem already gives for every increasing function, and states without proof the classification that settles the question. Since this is the last section of the chapter, it ends by summing up the chapter’s moves and listing the promises the chapter has paid.

Throughout, $F=\mathbb{R}$ or $F=\mathbb{C}$, and every matrix whose eigenvalues are indexed is Hermitian, with eigenvalues ${\lambda }_1(\mathbf{A})\ge \cdots \ge {\lambda }_n(\mathbf{A})$ indexed decreasingly and repeated according to multiplicity. The letter $I$ denotes an interval of real numbers (open, closed, or half-open, bounded or not). It is plain italic, and it is not the bold identity matrix $\mathbf{I}$. A function $f:I\to \mathbb{R}$ is increasing when $s\le t$ implies $f(s)\le f(t)$; strictness is not required. Every result below holds for either choice of $F$, with the same proof. No analysis is used anywhere in this section except in the last part, which states a theorem without proving it.

17.11.1. Functions of a Hermitian matrix

Chapter 12 §08 already applies a function to a matrix, and nothing needs rebuilding. Let $\mathbf{A}\in M_n(F)$ be Hermitian. The operator $\mathbf{x}\mapsto \mathbf{A}\mathbf{x}$ on $F^n$ has an orthonormal basis of eigenvectors (Corollary 12.4.3, Corollary 12.5.2), so it has a spectral resolution $$\mathbf{A}={\mu }_1{\mathbf{P}}_1+\cdots +{\mu }_k{\mathbf{P}}_k$$ with ${\mu }_1,\dots ,{\mu }_k$ the distinct eigenvalues and ${\mathbf{P}}_j$ the orthogonal projection onto the eigenspace of ${\mu }_j$ (Definition 12.8.1, Theorem 12.8.1). For any function $f$ defined on $\mathrm{spec}(\mathbf{A})$, Definition 12.8.2 sets $$f(\mathbf{A}):=f({\mu }_1){\mathbf{P}}_1+\cdots +f({\mu }_k){\mathbf{P}}_k.$$ In this section $f$ is always defined on an interval $I$ containing $\mathrm{spec}(\mathbf{A})$, and $f(\mathbf{A})$ is formed from the restriction of $f$ to the spectrum: the values of $f$ off the spectrum play no part. Four facts from Theorem 12.8.3 are used constantly.

• Polynomials give the expected answer (part (a)): $t\mapsto \alpha +\beta t$ gives $\alpha \mathbf{I}+\beta \mathbf{A}$, and $t\mapsto t^2$ gives ${\mathbf{A}}^2$.
• Sums and products go through (part (b)): $(f+g)(\mathbf{A})=f(\mathbf{A})+g(\mathbf{A})$, $(cf)(\mathbf{A})=cf(\mathbf{A})$, and $(fg)(\mathbf{A})=f(\mathbf{A})g(\mathbf{A})$.
• Real-valued $f$ gives a Hermitian $f(\mathbf{A})$ (part (c)). This is what lets $f(\mathbf{A})$ be compared with anything in the Loewner order.
• The eigenvectors do not change (part (d)): $f(\mathbf{A})$ has the same orthonormal eigenbasis as $\mathbf{A}$.

Two instances have names already. If $0\notin \mathrm{spec}(\mathbf{A})$, then $t\mapsto 1/t$ gives ${\mathbf{A}}^{-1}$, as Chapter 12 §08 records right after Definition 12.8.2. If $\mathbf{A}⪰\mathbf{O}$, then $t\mapsto t^{1/2}$ gives the positive square root ${\mathbf{A}}^{1/2}$ of Theorem 13.2.1, which Chapter 13 §02 constructed in exactly this way.

17.11.2. Operator monotone and operator convex functions

Chapter 13 §05 introduced the Loewner order $\mathbf{A}⪰\mathbf{B}$, meaning $\mathbf{A}-\mathbf{B}⪰\mathbf{O}$ (Definition 13.5.1). With the functional calculus available, the natural question is which functions $f$ carry $\mathbf{A}⪰\mathbf{B}$ to $f(\mathbf{A})⪰f(\mathbf{B})$.

The obvious guess is “the increasing ones”, since that is the answer for numbers. It is wrong. The function $t\mapsto t^2$ is increasing on $[0,\mathrm{\infty })$, and Example (Squaring does not preserve the Loewner order) gives Hermitian $2\times 2$ matrices $\mathbf{A}⪰\mathbf{B}⪰\mathbf{O}$ with ${\mathbf{A}}^2⪰̸{\mathbf{B}}^2$. So the property we want is stronger than being increasing, and it needs its own name.

A function is operator monotone when applying it to Hermitian matrices never breaks the Loewner order, at every size at once.

In words: the first condition takes any two Hermitian matrices of the same size with spectra inside the domain of $f$. If the first dominates the second, applying $f$ must preserve that. The second condition is the matrix version of “the chord lies above the graph”. Both are infinitely many conditions, one for each size $n$. A function that passes the test for $2\times 2$ matrices has not yet passed it for $3\times 3$ ones. The clause “for every $n$” is why the notion is hard, and also why it is interesting.

Everything in the definition makes sense. Since $\mathrm{spec}(\mathbf{A})\subseteq I$, the matrix $f(\mathbf{A})$ is defined, and it is Hermitian because $f$ is real-valued (Theorem 12.8.3 (c)). So $f(\mathbf{A})⪰f(\mathbf{B})$ is a statement about two Hermitian matrices, as Definition 13.5.1 requires. In (b) one more thing needs checking: the spectrum of $\mathbf{C}=\theta \mathbf{A}+(1-\theta )\mathbf{B}$ must lie in $I$, or $f(\mathbf{C})$ would not be defined. The matrix $\mathbf{C}$ is Hermitian. Let $\mathbf{x}$ be a unit eigenvector of $\mathbf{C}$ for the eigenvalue $\mu$. Then $$\mu =⟨\mathbf{C}\mathbf{x},\mathbf{x}⟩=\theta ⟨\mathbf{A}\mathbf{x},\mathbf{x}⟩+(1-\theta )⟨\mathbf{B}\mathbf{x},\mathbf{x}⟩.$$ By Lemma 13.5.2, $⟨\mathbf{A}\mathbf{x},\mathbf{x}⟩$ lies in $[{\lambda }_n(\mathbf{A}),{\lambda }_1(\mathbf{A})]$. That interval is contained in $I$, because its endpoints are eigenvalues of $\mathbf{A}$ and $I$ is an interval. Likewise $⟨\mathbf{B}\mathbf{x},\mathbf{x}⟩\in I$. So $\mu$ is a convex combination of two points of the interval $I$, and it therefore lies in $I$.

The smallest size already says something, and the proof is three lines.

The converse is false, and the failure is the reason this section exists.

17.11.3. The examples we can prove

Four families are within reach, and the first is the simplest. It includes a degenerate case, the constant functions, which respect every order because they flatten everything to one matrix.

The second example is the one that behaves like the square root: $t\mapsto -1/t$. For numbers, $a\ge b>0$ gives $-1/a\ge -1/b$ because one may divide by the product $ab$. For matrices that division is unavailable, since $\mathbf{A}$ and $\mathbf{B}$ need not commute. Chapter 13 §05 already found the way round.

Two things make the whitening work, and they are worth separating. First, once one of the two matrices is $\mathbf{I}$, the comparison $\mathbf{C}⪰\mathbf{I}$ can be settled eigenvalue by eigenvalue, because $\mathbf{I}$ commutes with everything. Second, inversion gets along with congruence: $({\mathbf{S}}^{\ast }\mathbf{M}\mathbf{S}{)}^{-1}={\mathbf{S}}^{-1}{\mathbf{M}}^{-1}({\mathbf{S}}^{-1}{)}^{\ast }$ is again a congruence, and that is what carries the conclusion back from $\mathbf{C}$ to $\mathbf{A}$. For a general $f$ there is no such identity, since $f({\mathbf{S}}^{\ast }\mathbf{M}\mathbf{S})$ is usually not a congruence of $f(\mathbf{M})$, so the argument does not simply transfer to other functions.

To build more examples out of these, one needs to know which ways of combining functions preserve the property.

Combining the last two propositions produces a whole one-parameter family. It will reappear in the final part of the section as the building block of every operator monotone function.

The fourth example is the one that started the section, and it is already proved. By Proposition 13.5.4, $\mathbf{A}⪰\mathbf{B}⪰\mathbf{O}$ implies ${\mathbf{A}}^{1/2}⪰{\mathbf{B}}^{1/2}$. A Hermitian matrix has spectrum in $[0,\mathrm{\infty })$ exactly when it is $⪰\mathbf{O}$ (Theorem 13.1.4 (b)), and ${\mathbf{A}}^{1/2}$ is $f(\mathbf{A})$ for $f(t)=t^{1/2}$. So Chapter 13 §05’s proposition says precisely that $t\mapsto t^{1/2}$ is operator monotone on $[0,\mathrm{\infty })$. It is not re-proved here. Its proof was a different kind of argument, an eigenvector of ${\mathbf{A}}^{1/2}-{\mathbf{B}}^{1/2}$ pushed to a contradiction, and it too never needs $\mathbf{A}$ and $\mathbf{B}$ to commute.

A nonexample by minimal change. The function $t\mapsto t=t^1$ is operator monotone on $[0,\mathrm{\infty })$ by Proposition 17.11.2. Raise the exponent from $1$ to $2$. The function is still increasing, still a polynomial, and still satisfies the case $n=1$. What fails is the clause “for every $n$”, already at $n=2$, with the witness of Example (Squaring does not preserve the Loewner order). Lowering the exponent from $1$ to $\frac{1}{2}$ instead keeps the property. Among the powers, then, the dividing line lies somewhere around the exponent $1$. Chapter 13 §05 said where it lies exactly, and the last part of this section returns to that.

Why “for every $n$”. Each size imposes its own condition, and the conditions differ. At $n=1$, $t\mapsto t^2$ passes on $[0,\mathrm{\infty })$ and at $n=2$ it fails. A definition at one fixed size would admit functions that break at the next size, and it would not say what the functional calculus does to matrices in general. Requiring every size at once makes the class small enough to classify completely, which is what Loewner’s theorem at the end of this section does. The name comes from the same place as the name of the order: both are Loewner’s.

17.11.4. Eigenvalues move monotonically for every increasing function

Operator monotonicity is a strong requirement, and a weaker one comes for free. §02 proved that the whole ordered list of eigenvalues respects the Loewner order: by Corollary 17.2.4, $\mathbf{A}⪰\mathbf{B}$ implies ${\lambda }_i(\mathbf{A})\ge {\lambda }_i(\mathbf{B})$ for every $i$. That paid the debt Chapter 13 §05 recorded after Theorem 13.5.3, and it holds with no hypothesis on commuting. Composing with an increasing $f$ keeps it, once one knows what $f$ does to the ordered list of eigenvalues.

The lemma needs $f$ increasing only for the order of the list. For a general $f$, the numbers $f({\lambda }_i(\mathbf{A}))$ are still the eigenvalues of $f(\mathbf{A})$, but they may have to be re-sorted. Two lines now give the eigenvalue-level statement.

So the squaring map, which is not operator monotone, is nevertheless monotone on eigenvalues. The Chapter 13 witness shows both at once.

So for every increasing $f$, the eigenvalues of $f(\mathbf{A})$ dominate those of $f(\mathbf{B})$ one index at a time. What operator monotonicity adds is that the matrices are ordered, and the example shows that this is strictly more.

17.11.5. Operator convexity, and where the two notions part

Operator convexity is the second notion of Definition 17.11.1, and one might expect it to behave like operator monotonicity. It does not: the function that fails to be operator monotone turns out to be operator convex.

The contrast is now complete, and it runs both ways.

• $t\mapsto t^2$ is operator convex but not operator monotone on $[0,\mathrm{\infty })$: Proposition 17.11.8 and Example (Squaring does not preserve the Loewner order).
• $t\mapsto -1/t$ is operator monotone but not operator convex on $(0,\mathrm{\infty })$: Proposition 17.11.3, and at $n=1$ the scalar function $-1/t$ is not even convex, since with $a=1$, $b=3$, and $\theta =\frac{1}{2}$ the value at the midpoint is $-\frac{1}{2}$, above the chord’s $-\frac{2}{3}$.

So neither notion implies the other. Each implies its scalar version (increasing, convex) by taking $n=1$, and they strengthen those versions in different directions. Exercise C2 adds that $t\mapsto 1/t$ is operator convex on $(0,\mathrm{\infty })$, so $-1/t$ is operator monotone and operator concave there.

17.11.6. What is not proved here

The question the section opened with has a complete answer, and this chapter does not prove it. Here is the statement, so that the reader knows what the answer looks like and which of the facts above are pieces of it.

We do not prove this theorem here. Its proof needs complex analysis and some measure theory, which lie outside Chapter 16’s list (A1)–(A6); Chapter 21 proves it. Nothing in this chapter depends on it.

Read (ii) against what was proved above. The functions it superposes are $t\mapsto -1/(t+s)$ for $s\ge 0$, the translates of $-1/t$ that Corollary 17.11.5 showed to be operator monotone. The constants $s/(1+s^2)$ do not depend on $t$ and are there only to make the integral converge. The term $\alpha +\beta t$ is Proposition 17.11.2. So the theorem says that the examples of this section are, in a precise sense, all the examples there are: every operator monotone function on $(0,\mathrm{\infty })$ is an increasing affine function plus a nonnegative superposition of shifted reciprocals. The easy direction, (ii) $\Rightarrow$ (i), is essentially Proposition 17.11.4 applied to an integral rather than a finite sum. What this book lacks is the analysis that carries the Loewner order through an integral, which is not on the list (A1)–(A6) in Chapter 16’s introduction. The hard direction, (i) $\Rightarrow$ (ii), is the classification itself.

Condition (iii) answers the opening question in one line. For $z$ in the upper half-plane, $z^{1/2}$ (the branch that is positive on $(0,\mathrm{\infty })$) halves the argument of $z$, so its argument stays between $0$ and $\pi /2$ and $z^{1/2}$ stays in the upper half-plane. The square doubles the argument, which can pass $\pi$. The square root keeps the upper half-plane, and the square folds it over.

The same test settles the powers $t\mapsto t^p$. For $0<p<1$, the function $z^p$ multiplies the argument by $p$ and keeps the upper half-plane, and indeed for $t>0$ (a standard integral, not proved here) $$t^p=\frac{\sin p\pi }{\pi }{\int }_0^{\mathrm{\infty }}{s}^{p-1}\frac{t}{t+s}\mathrm{d}s,$$ a positive superposition of the functions $t/(t+s)$ of Corollary 17.11.5. For $p>1$, the argument of $z^p$ passes $\pi$ for some $z$ in the upper half-plane, and the test fails. This is the dividing line at exponent $1$ that Chapter 13 §05 announced. Both halves, that $t^p$ is operator monotone on $(0,\mathrm{\infty })$ for every $p\in (0,1)$ and that it is not for any $p>1$, are proved in Chapter 21, with Loewner’s theorem. What is proved so far is the case $p=\frac{1}{2}$ (Proposition 13.5.4) and, by applying it repeatedly, the cases $p=1/2^k$ (exercise C1).

17.11.7. Summary and transfer

The chapter began with a list of eigenvalues that the book had been able to order since Chapter 13 without being able to describe. It ends with each eigenvalue described as an optimization, and with a set of comparison theorems that follow from that description. Five moves did the work, and each transfers beyond this chapter.

• The dimension count. Two subspaces of an $n$-dimensional space whose dimensions add to more than $n$ share a nonzero vector. This one line drives Courant–Fischer (Theorem 17.2.3) and Weyl’s inequalities (§03) directly, and through Courant–Fischer everything after them. Transfer: whenever an extremal problem over vectors has to be compared with a fixed subspace, count dimensions before computing anything.
• Min–max. Describe a quantity as a minimum over subspaces of a maximum over vectors, and prove it with two inequalities of different characters: one exhibits a witness subspace and computes, and the other takes an arbitrary subspace and intersects it with a span of eigenvectors. Applying the result to $-\mathbf{A}$ gives the dual form for free. Transfer: a description that names no eigenvector is what makes two different matrices comparable. The same shape describes singular values (§09) and inertia (§10).
• Compress and interlace. Restricting a Hermitian form to a subspace gives a smaller Hermitian matrix whose eigenvalues are trapped by the original’s: compressing to a $k$-dimensional subspace puts ${\lambda }_i$ of the piece between ${\lambda }_i$ and ${\lambda }_{i+n-k}$ of the whole, which are consecutive when $k=n-1$. Deleting a row and column (§04), compressing to an orthonormal frame (§06) and deleting a row or a column of a rectangular matrix (§09) are all this move; adding a rank-one term (§05) reaches the same interlaced picture by a different route, the Loewner order together with Weyl’s rank bound. Transfer: to bound the spectrum of a piece, look for the whole of which it is a compression.
• Perturb in order. Compare ${\lambda }_i(\mathbf{A}+\mathbf{E})$ with ${\lambda }_i(\mathbf{A})$ index by index, never “each eigenvalue with its nearest neighbor”. For Hermitian perturbations the Lipschitz constant is $1$ in the spectral norm, independent of $n$, of the gaps, and of the eigenvectors (§03). For singular values it needs no hypothesis on $\mathbf{E}$ at all (§09). Transfer: where the Jordan block of Chapter 16 §07 showed that eigenvalues of general matrices can move like ${\epsilon }^{1/k}$, the Hermitian and singular-value problems are perfectly conditioned. Reduce a question to one of them whenever possible.
• Majorize. When the right comparison is between sums of the largest entries rather than individual ones, compare partial sums of decreasing rearrangements. The diagonal of a Hermitian matrix is majorized by its spectrum, and by the Schur–Horn theorem majorization describes the possible diagonals exactly (§08); and the Ky Fan and Lidskii sums (§§06–07) are of the same type. Transfer: a family of partial-sum inequalities indexed by $k$, together with equality of the totals, is a majorization in disguise, and writing it as one lets the whole family be used at once.

Two supporting moves recur under these five: apply the result to $-\mathbf{A}$, which turns every max–min into a min–max and every top-$k$ statement into a bottom-$k$ one, and reduce singular values to eigenvalues, through ${\mathbf{A}}^{\ast }\mathbf{A}$ or the Hermitian dilation. This section also reused a move from Chapter 13 §05, whiten, then compare: use a congruence to make one of the two matrices $\mathbf{I}$. It works for $t\mapsto -1/t$ because inversion turns a congruence into a congruence, and not for a general $f$.

The chapter also paid a series of promises made in earlier chapters, some of them long ago.

Four earlier theorems also received second proofs from the variational side, each of which shows something the first proof did not. The positivity of principal submatrices of a positive semidefinite matrix now follows from interlacing (§04, Deleting a row and a column). Hadamard’s inequality Theorem 13.7.3 follows from AM–GM after a diagonal scaling, with Schur’s theorem supplying the equal totals (§08, The diagonal and the spectrum). Sylvester’s law of inertia Theorem 14.5.1 is now a statement about the dimensions of subspaces, with no basis in it (§10, Inertia, again and additively). And the block test Theorem 13.6.1 is now a consequence of inertia additivity (§10).

Some things the chapter did not do. Bounds for the eigenvalues of non-Hermitian matrices under perturbation, beginning with Bauer–Fike, are Chapter 20’s. Chapter 21 has Loewner’s theorem with the powers $t^p$, Schur-concave functions, the strongest form of Lidskii’s inequality, and Eckart–Young in every unitarily invariant norm. The variational principle itself, though, is now a tool the book can use without comment: an eigenvalue of a Hermitian matrix is a min–max, and comparison theorems follow from that.

17.11.8. Exercises

A. Check your understanding

B. Practice