## The Rayleigh Quotient

The Rayleigh quotient arises from attempting to optimise quadratic forms subject to constraints.
Suppose we have a quadratic form
$\langle \mathbf{x}, A \mathbf{x} \rangle$
subject to the constrain
$\langle \mathbf{x}, \mathbf{x} \rangle =1$
where
$A$
is an
$n \times n$
Hermitian matrix, so that
$A$
is equal to its own complex conjugate transpose.
The Rayleigh quotient is defined as
$\rho = \rho( \mathbf{x})= \frac{\langle \mathbf{x},A \mathbf{x} \rangle}{\langle \mathbf{x}, \mathbf{x} \rangle}$
(1)
Furthermore, if
$A$
is Hermitian with eigenvalues
$\lambda_1 \leq \lambda_2 \leq ... \leq \lambda_n$
and associated eigenvectors
$\mathbf{v}_1, \mathbf{v}_2,..., \mathbf{v}_n$
then
$\lambda_1 \leq \rho( \mathbf{x}) \leq \lambda_n$
and
$\lambda_1 = min(\rho(\mathbf{x}))= \rho(\mathbf{x}_1)$

We can use (1) to find approximate eigenvalues. Let
$\mathbf{x}_i$
be an eigenvector associated with the eigenvalue
$\lambda_i$
, which must be real since the matrix is Hermitian.
Define
$\mathbf{x}=\mathbf{x}_i + \epsilon \mathbf{z}, \: \| \epsilon \| \ll 1$
.
Then
$\rho(\mathbf{x})= \lambda_i+[ \rho(\mathbf{z})- \lambda_i ] \frac{\langle \mathbf{z}, \mathbf{z} \rangle}{\langle \mathbf{x}, \mathbf{x} \rangle} \| \epsilon \|^2$

Suppose
$A= \left( \begin{array}{ccc} 1.7 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{array} \right)$
.
\begin{aligned} \rho (\mathbf{x}) &= \frac{(x_1,x_2,x_3)\left( \begin{array}{ccc} 1.7 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{array} \right) \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}}{\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} \cdot \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}} \\ &= \frac{1.7x_1^2+2x_2^2+2x_3^2-2x_1x_2-2x_2x_3}{x_1^2+x_2^2+x_3^2} \end{aligned}

The eigenvector(s) associated with the smallest eigenvalue will have components of the same sign. Let
$\mathbf{v}_1= \begin{pmatrix}1\\1\\1\end{pmatrix}$
, then
$\rho(\mathbf{v}_1)=0.57$
.
If
$\mathbf{v}_1= \begin{pmatrix}1\\2\\1\end{pmatrix}$
, then
$\rho(\mathbf{v}_1)=0.62$
.
The estimates are upper bounds on
$\lambda_1$
so the best guess so far for
$\lambda_1$
is 0.57.
In fact
$\lambda_1=0.5$
with eigenvector
$\mathbf{v}_1= \begin{pmatrix}1\\1.2\\0.8\end{pmatrix}$
.