• search direction
• descent direction
• direction of strict descent
• direction of steepest descent
• the Cauchy direction
• The Newton direction
• step length , step size
• Curry step length
• Armijo-Goldstein inequality
• backtracking line search
• The Basic Backtracking Algorithm
• Convergence Theorem for the backtracking line search
• the weak Wolfe conditions
• the strong Wolfe conditions
• the Goldstein conditions
• The Bisection Method for the Weak Wolf Conditions

• Steepest Descent direction
• Newton's method for solving equations
• Newton-Like methods
• Q-convergence rates
• linear, superlinear, and quadratic convergence
• Newton's method for minimization
• The rate of convergence of Newton's method
• Matrix secant equation
• Broyden's method (both direct and inverse updates)
• the BFGS update (both direct and inverse updates)

• LU factorization
• Cholesky Factorization
• QR Factorization

• Goldstein conditions
• Wolfe conditions
• Basic Backtracking line search
• Bisection method for the weak Wolfe conditions

• Steepest descent
• Newton's method for equations and minimization
• Broyden's method
• BFGS

• the LU, Cholesky, and QR factorizations

Skills to Master:

• implementing the backtracking and weak Wolfe bisection line searches.
• implementing Newton and Newton-like methods for both equation solving and optimization.
• Implementing Broyden and BFGS in Matlab
• understand how to implement the Matlab command "chol"

Quiz:

• The quiz will consist of 2 questions. The first question will be related to the vocabulary words from the notes on Line Search Methods and Search Direction only (i.e. no other vocabulary words will be asked). In the second question you will be asked to either apply Newton's method to a given problem (either for equation solving or minimization), and/or you you will be given a convergent sequence of real numbers and asked to determine if it is linearly, superlinearly, or quadratically convergent.