Stat 567:Statistical Reliability
Seventeenth Class (November 5, 1997)

Nonlinear Programing:
Multivariate Optimization Without Constraints

Class Objectives:

• Learn about some optimization techniques used for maximizing likelihood functions
• Understand how JMP's Nonlinear Fit platform performs optimizations

Homework Assignments:

Class Outline and Main Points:

• Optimization is frequently equivalent to finding the zeros of functions (namely, derivatives).
• Newton's method in the univariate case
  • Formula
  • Derivation
  • Convergence properties at different starting points
• Definitions
  • Object function
  • Local maximum
  • Global maximum
  • Gradient vectors
  • Hessian matrix
  • Symmetric matrix
  • Negative definite and negative semi-definite matrices
• Theorems
  • How to tell that a matrix is negative definite or negative semi-definite using principal minors
  • Local maxima, gradients, and negative definite Hessian matrices
• The method of steepest ascent
• The Newton-Raphson method
  • Properties
  • Derivation
• Newton-Raphson method and maximum likelihood
  • Relationship of Hessian matrix to Information matrix
  • When is the Newton-Raphson method effective for maximum likelihood estimation
• Other optimization techniques and maximum likelihood
  • Scoring method
  • Gauss-Newton method

Study Questions

• Why is it important to have good initial values when using JMP's Nonlinear Fit platform?
• Why do I say that the ancient Sumerians used the Newton Method to find the square root of a number?
• In what direction does the gradient vector points?

Do you have something to tell me?