Revising for an exam on machine learning and pattern recognition bitsandpieces on Friday and I came across a paper titled ‘An Introduction to the Conjugate Gradient Method Without the Agonizing Pain‘. Which is awesome.
The Conjugate Gradient Method is the most prominent iterative method for solving sparse systems of linear equations. Unfortunately, many textbook treatments of the topic are written with neither illustrations nor intuition, and their victims can be found to this day babbling senselessly in the corners of dusty libraries. For this reason, a deep, geometric understanding of the method has been reserved for the elite brilliant few who have painstakingly decoded the mumblings of their forebears. Nevertheless, the Conjugate Gradient Method is a composite of simple, elegant ideas that almost anyone can understand. Of course, a reader as intelligent as yourself will learn them almost effortlessly.
Its keywords are conjugate gradient method, preconditioning, convergence analysis and agonizing pain, and it has a chapter called ‘Eigen do it if I try‘ which (brilliant and awful title aside) is the first intuitive explanation of eigenvectors I’ve read, hurrah.