The linear algebra of block quasi-newton algorithms

The quasi-Newton family of algorithms for minimizing functions and solving systems of nonlinear equations has achieved a great deal of computational success and forms the core of many software libraries for solving these problems. In this work we extend the theory of the quasi-Newton algorithms to the block case, in which we minimize a collection of functions having a common Hessian matrix, or we solve a collection of nonlinear equations having a common Jacobian matrix. This paper focuses on the linear algebra: update formulas, positive definiteness, least-change secant properties, relation to block conjugate gradient algorithms, finite termination for quadratic function minimization or solving linear systems, and the use of the quasi-Newton matrices as preconditioners.