The H-norm

Perhaps the ideal trust region would be in the H-norm, for which

(12.3) |

and where the absolute value is defined by , where is a diagonal matrix constituted by the eigenvalues of and where is an orthonormal matrix of the associated eigenvectors and where the absolute value of the diagonal matrix is simply the matrix formed by taking absolute values of its entries.

This norm reflects the proper scaling of the underlying problem - directions for which the model is changing fastest, and thus directions for which the model may differ most from the true function are restricted more than those for which the curvature is small.

The eigenvalue decomposition is extremely expensive to compute. A solution, is to consider the less expensive symmetric, indefinite factorization ( is a permutation matrix, is unit lower triangular, is block diagonal with blocks of size at most 2). We will use with computed by taking the absolute values of the 1 by 1 pivots and by forming an independent spectral decomposition of each of the 2 by 2 pivots and reversing the signs of any resulting negative eigenvalues.

For more information see [CGT00g].