[113] P. E. Gill and W. Murray, Newton-type methods for unconstrained and linearly con-

strained optimization, Math. Programming, 28 (1974), pp. 311“350.

, Safeguarded Steplength Algorithms for Optimization Using Descent Methods,

[114]

Tech. Rep. NAC 37, National Physical Laboratory Report, Teddington, England, 1974.

, Non-linear least squares and nonlinearly constrained optimization, in Numerical

[115]

Analysis, Lecture Notes in Mathematics 506, Springer-Verlag, Berlin, 1976.

, Algorithms for the solution of the nonlinear least-squares problem, SIAM J. Numer.

[116]

Anal., 15 (1978), pp. 977“992.

[117] P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization, Academic Press,

London, 1981.

[118] P. Gilmore, An Algorithm for Optimizing Functions with Multiple Minima, Ph.D. thesis,

North Carolina State University, Raleigh, 1993.

, IFFCO: Implicit Filtering for Constrained Optimization, Tech. Rep. CRSC-TR93-

[119]

7, Center for Research in Scienti¬c Computation, North Carolina State University, May

1993. Available by anonymous ftp from math.ncsu.edu in pub/kelley/iffco/ug.ps.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

168 BIBLIOGRAPHY

[120] P. Gilmore and C. T. Kelley, An implicit ¬ltering algorithm for optimization of functions

with many local minima, SIAM J. Optim., 5 (1995), pp. 269“285.

[121] P. A. Gilmore, S. S. Berger, R. F. Burr, and J. A. Burns, Automated optimization

techniques for phase change piezoelectric ink jet performance enhancement. in Proc.

IS&T™s NIP 13: 1997 International Conference on Digital Printing Technologies, Society

for Imaging Science and Technology, 1997, pp. 716“721.

[122] R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer-Verlag,

New York, 1984.

[123] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning,

Addison“Wesley, Reading, MA., 1989.

[124] D. Goldfarb, A family of variable metric methods derived by variational means, Math.

Comp., 24 (1970), pp. 23“26.

[125] A. A. Goldstein, Constructive Real Analysis, Harper and Row, New York, 1967.

[126] G. H. Golub and V. Pereyra, The differentiation of pseudo-inverses and nonlinear least

squares problems whose variables separate, SIAM J. Numer. Anal., 10 (1973), pp. 413“

432.

[127] G. H. Golub and C. G. Van Loan, Matrix Computations, Johns Hopkins University

Press, Baltimore, MD, 1983.

[128] A. Greenbaum, Iterative Methods for Solving Linear Systems, SIAM, Philadelphia, 1997.

[129] A. Griewank, On automatic differentiation, in Mathematical Programming: Recent De-

velopments and Applications, M. Iri and K. Tanabe, eds., Kluwer, Dordrecht, the Nether-

lands, 1989, pp. 83“108.

[130] A. Griewank and G. F. Corliss, eds., Automatic Differentiation of Algorithms: Theory,

Implementation, and Application, SIAM, Philadelphia, 1991.

[131] A. Griewank and P. L. Toint, Local convergence analysis for partitioned quasi-Newton

updates, Numer. Math., 39 (1982), pp. 429“448.

, On the unconstrained optimization of partially separable functions, in Nonlinear

[132]

Optimization, M. J. D. Powell, ed., Academic Press, London, 1982.

, Partitioned variable metric methods for large sparse optimization problems, Nu-

[133]

mer. Math., 39 (1982), pp. 119“137.

[134] L. Grippo and S. Lucidi, A globally convergent version of the Polak-Ribi` re conjugate

e

gradient method, Math. Programming, 78 (1997), pp. 375“392.

[135] W. A. Gruver and E. Sachs, Algorithmic Methods in Optimal Control, Pitman, London,

1980.

[136] W. Hackbusch, On the fast solving of parabolic boundary control problems, SIAM J.

Control Optim., 17 (1979), pp. 231“244.

, Optimal H p,p/2 error estimates for a parabolic Galerkin method, SIAM J. Numer.

[137]

Anal., 18 (1981), pp. 681“692.

, Multi-Grid Methods and Applications, Springer Ser. Comput. Math. 4, Springer-

[138]

Verlag, New York, 1985.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

BIBLIOGRAPHY 169

[139] W. W. Hager, Rates of convergence for discrete approximations to unconstrained optimal

control problems, SIAM J. Numer. Anal., 13 (1976), pp. 449“472.

[140] M. Heinkenschloss, M. Ulbrich, and S. Ulbrich, Superlinear and Quadratic Conver-

gence of Af¬ne Scaling Interior-Point Newton Methods for Problems with Simple Bounds

and Without Strict Complementarity Assumption, Tech. Rep. TR97-30, Department of

Computational and Applied Mathematics, Rice University, Houston, TX, December 1997.

[141] M. R. Hestenes and E. Steifel, Methods of conjugate gradient for solving linear sys-

tems, J. Res. Nat. Bureau Standards, 49 (1952), pp. 409“436.

[142] M. R. Hoare and P. Pal, Physical cluster mechanics: Statics and energy surfaces for

monoatomic systems, Adv. Phys., 20 (1971), pp. 161“196.

[143] J. H. Holland, Genetic algorithms and the optimal allocation of trials, SIAM J. Comput.,

2 (1973), pp. 88“105.

, Adaption in Natural and Arti¬cial Systems, University of Michigan Press, Ann

[144]

Arbor, MI, 1975.

[145] R. Hooke and T. A. Jeeves, “Direct search" solution of numerical and statistical prob-

lems, J. Assoc. Comput. Mach., 8 (1961), pp. 212“229.

[146] R. Horst, P. M. Pardolos, and N. V. Thoai, Introduction to Global Optimization,

Kluwer Academic Publishers, Dordrecht, the Netherlands, 1995.

[147] W. Huyer and A. Neumaier, Global Optimization by Multilevel Coordinate Search,

Institut f¨ r Mathematik, Universit¨ t Wien, 1997, preprint.

u a

[148] D. M. Hwang and C. T. Kelley, Convergence of Broyden™s method in Banach spaces,

SIAM J. Optim., 2 (1992), pp. 505“532.

[149] G. Jasen, Investment dartboard: Pros and dart throwers both lose money, Your Money

Matters, Wall Street Journal, May 7, 1997.

[150] D. R. Jones, C. C. Perttunen, and B. E. Stuckman, Lipschitzian optimization without

the Lipschitz constant, J. Optim. Theory Appl., 79 (1993), pp. 157“181.

[151] L. Kantorovich and G. Akilov, Functional Analysis, 2nd ed., Pergamon Press, New

York, 1982.

[152] R. B. Kearfott, Rigorous Global Search: Continuous Problems, Kluwer, Dordrecht,

the Netherlands, 1966.

[153] C. T. Kelley, Identi¬cation of the support of nonsmoothness, in Large Scale Optimiza-

tion: State of the Art, W. W. Hager, D. W. Hearn, and P. Pardalos, eds., Kluwer Academic

Publishers B.V., Boston, 1994, pp. 192“205.

, Iterative Methods for Linear and Nonlinear Equations, SIAM, Philadelphia, 1995.

[154]

, Detection and remediation of stagnation in the Nelder-Mead algorithm using a

[155]

suf¬cient decrease condition, SIAM J. Optim., 10 (1999), pp. 43“55.

[156] C. T. Kelley, C. T. Miller, and M. D. Tocci, Termination of Newton/chord iterations

and the method of lines, SIAM J. Sci. Comput., 19 (1998), pp. 280“290.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

170 BIBLIOGRAPHY

[157] C. T. Kelley and E. W. Sachs, Applications of quasi-Newton methods to pseu-

doparabolic control problems, in Optimal Control of Partial Differential Equations II”

Theory and Applications, Birkh¨ user, Basel, 1987.

a

, Quasi-Newton methods and unconstrained optimal control problems, SIAM J.

[158]

Control Optim., 25 (1987), pp. 1503“1516.

, A pointwise quasi-Newton method for unconstrained optimal control problems,

[159]

Numer. Math., 55 (1989), pp. 159“176.

, Pointwise Broyden methods, SIAM J. Optim., 3 (1993), pp. 423“441.

[160]

, Multilevel algorithms for constrained compact ¬xed point problems, SIAM J. Sci.

[161]

Comput., 15 (1994), pp. 645“667.

, Local convergence of the symmetric rank-one iteration, Computational Optimiza-

[162]

tion and Applications, 9 (1998), pp. 43“63.

, A trust region method for parabolic boundary control problems, SIAM J. Optim.,

[163]

9 (1999), pp. 1064“1081.

[164] C. T. Kelley, E. W. Sachs, and B. Watson, A pointwise quasi-Newton method for

unconstrained optimal control problems, II, J. Optim. Theory Appl., 71 (1991), pp. 535“

547.

[165] H. F. Khalfan, R. H. Byrd, and R. B. Schnabel, A theoretical and experimental study

of the symmetric rank-one update, SIAM J. Optim., 3 (1993), pp. 1“24.

[166] S. Kirkpatrick, C. D. Geddat, and M. P. Vecchi, Optimization by simulated annealing,

Science, 220 (1983), pp. 671“680.

[167] J. R. Koehler and A. B. Owen, Computer experiments, in Handbook of Statistics, vol.

13, S. Shosh and C. R. Rao, eds., Elsevier, New York, 1996, pp. 261“308.

[168] J. Kostrowicki and L. Piela, Diffusion equation method of global minimization: Per-

formance for standard test functions, J. Optim. Theory Appl., 71 (1991), pp. 269“284.

[169] J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, Convergence Properties

of the Nelder-Mead Simplex Algorithm in Low Dimensions, SIAM J. Optim., 9 (1998),

pp. 112“147.

[170] E. B. Lee and L. Markus, Foundations of Optimal Control Theory, John Wiley, New

York, London, Sydney, 1967.

[171] C. Lemar´ chal, A view of line searches, in Optimization and Optimal Control, Auslander,

e

Oettli, and Stoer, eds., Lecture Notes in Control and Information Sciences 30, Springer-

Verlag, Berlin, 1981, pp. 59“78.

[172] K. Levenberg, A method for the solution of certain nonlinear problems in least squares,

Quart. Appl. Math., 4 (1944), pp. 164“168.

[173] R. M. Lewis and V. Torczon, Rank ordering and positive bases in pattern search algo-

rithms, Tech. Rep. 96-71, Institute for Computer Applications in Science and Engineering,

December, 1996.

, Pattern search algorithms for linearly constrained minimization, SIAM J. Optim.,

[174]

9 (1999), pp. 1082“1099.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

BIBLIOGRAPHY 171

, Pattern search algorithms for linearly constrained minimization, SIAM J. Optim.,

[175]

10 (2000), pp. 917“941.

[176] D. C. Liu and J. Nocedal, On the limited memory BFGS method for large-scale opti-

mization, Math. Programming., 43 (1989), pp. 503“528.

[177] R. B. Long and W. C. Thacker, Data assimilation into a numerical equatorial ocean

model, part 2: Assimilation experiments, Dyn. Atmos. Oceans, 13 (1989), pp. 465“477.

[178] E. M. Lowndes, Vehicle Dynamics and Optimal Design, Ph.D. thesis, North Carolina

State University, Raleigh, 1998.

[179] S. Lucidi and M. Sciandrone, On the global convergence of derivative free methods

for unconstrained optimization, Universit` di Roma “La Sapienza”, Dipartimento di

a

Informatica e Sistemistica, 1997, reprint.

[180] D. G. Luenberger, Linear and Nonlinear Programming, Addison“Wesley, London,

1984.

[181] J. N. Lyness and C. B. Moler, Numerical differentiation of analytic functions, SIAM J.

Numer. Anal., 4 (1967), pp. 202“210.

[182] C. D. Maranas and C. A. Floudas, A global optimization method for Weber™s problem,

in Large Scale Optimization: State of the Art, W. W. Hager, D. W. Hearn, and P. Pardalos,

eds., Kluwer Academic Publishers B.V., Boston, 1994, pp. 259“293.

[183] D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters,

J. Soc. Indust. Appl. Math., 11 (1963), pp. 431“441.

[184] J. M. Martinez, Quasi-Newton methods for solving underdetermined nonlinear simul-

taneous equations, J. Comput. Appl. Math., 34 (1991), pp. 171“190.

[185] E. S. Marwil, Exploiting Sparsity in Newton-Type Methods, Ph.D. thesis, Cornell Uni-

versity, Ithaca, NY, 1978.

[186] H. Matthies and G. Strang, The solution of nonlinear ¬nite element equations, Internat.

J. Numer. Methods Engrg., 14 (1979), pp. 1613“1626.

[187] D. Q. Mayne and E. Polak, Nondifferential optimization via adaptive smoothing, J.

Optim. Theory Appl., 43 (1984), pp. 601“613.

[188] K. I. M. McKinnon, Convergence of the Nelder-Mead Simplex Method to a Non-

Stationary Point, SIAM J. Optim., 9 (1998), pp. 148“158.

[189] E. H. Moore, General Analysis, Memoirs of the American Philosophy Society, 1935. I.

[190] J. J. Mor´ , The Levenberg-Marquardt algorithm: Implementation and theory, in Numer-

e

ical Analysis, G. A. Watson, ed., Lecture Notes in Mathematics 630, Springer-Verlag,

Berlin, 1977, pp. 105“116.

, Trust regions and projected gradients, in System Modelling and Optimization,

[191]

Lecture Notes in Control and Information Sciences 113, Springer-Verlag, Berlin, 1988,

pp. 1“13.

[192] J. J. Mor´ and D. C. Sorensen, Computing a trust region step, SIAM J. Sci. Statist.

e

Comput., 4 (1983), pp. 553“572.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

172 BIBLIOGRAPHY

[193] J. J. Mor´ and D. J. Thuente, Line search algorithms with guaranteed suf¬cient de-

e

crease, ACM Trans. Math. Software, 20 (1994), pp. 286“307.

[194] J. J. Mor´ and G. Toraldo, On the solution of large quadratic programming problems

e

with bound constraints, SIAM J. Optim., 1 (1991), pp. 93“113.

[195] J. J. Mor´ and S. J. Wright, Optimization Software Guide, SIAM, Philadelphia, 1993.

e

[196] J. J. Mor´ and Z. Wu, Global continuation for distance geometry problems, SIAM J.

e

Optim., 7 (1997), pp. 814“836.

[197] W. Murray and M. L. Overton, Steplength algorithms for minimizing a class of non-

differentiable functions, Computing, 23 (1979), pp. 309“331.

[198] S. G. Nash, Newton-type minimization via the Lanczos method, SIAM J. Numer. Anal.,

21 (1984), pp. 770“788.

, Preconditioning of truncated-Newton methods, SIAM J. Sci. Statist. Comput., 6

[199]

(1985), pp. 599“616.

[200] L. Nazareth, A relationship between the BFGS and conjugate gradient algorithm and

its implications for new algorithms, SIAM J. Numer. Anal., 16 (1979), pp. 794“800.

, Conjugate gradient methods less dependent on conjugacy, SIAM Rev., 28 (1986),

[201]

pp. 501“511.

, A view of conjugate gradient-related algorithms for nonlinear optimization, in

[202]

Linear and Nonlinear Conjugate Gradient Methods, L. M. Adams and J. L. Nazareth,

eds., SIAM, Philadelphia, 1996, pp. 149“164.

[203] J. A. Nelder. Private Communication, 1998.

[204] J. A. Nelder and R. Mead, A simplex method for function minimization, Comput. J., 7

(1965), pp. 308“313.

[205] A. Neumaier, On convergence and restart conditions for a nonlinear conjugate gradient

method. Institut f¨ r Mathematik, Universit¨ t Wien, 1997, preprint.

u a

[206] J. Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35

(1980), pp. 773“782.

, Theory of algorithms for unconstrained optimization, Acta Numerica, 1 (1991),

[207]

pp. 199“242.

, Conjugate gradient methods and nonlinear optimization, in Linear and Nonlinear

[208]

Conjugate Gradient Methods, L. M. Adams and J. L. Nazareth, eds., SIAM, Philadelphia,

pp. 9“23.

[209] J. Nocedal and Y. Yuan, Combining Trust Region and Line Search Techniques, Tech.

Rep. OTC 98/04, Optimization Technology Center, Northwestern University, Chicago,

IL, 1998.

[210] J. A. Northby, Structure and binding of Lennard-Jones clusters: 13 ¤ n ¤ 147, J.

Chem. Phys., 87 (1987), pp. 6166“6177.

[211] J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in

Several Variables, Academic Press, New York, 1970.

Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.

BIBLIOGRAPHY 173

[212] R. Penrose, A generalized inverse for matrices, Proc. Cambridge Phil. Soc., 51 (1955),

pp. 406“413.

[213] L. R. Petzold, A description of DASSL: A differential/algebraic system solver, in Scien-

ti¬c Computing, R. S. Stepleman et al., ed., North“Holland, Amsterdam, 1983, pp. 65“68.

[214] S. A. Piyawskii, An algorithm for ¬nding the absolute extremum of a function, USSR

Comp. Math. and Math. Phys., 12 (1972), pp. 57“67.

[215] E. Polak and G. Ribi` re, Note sur la convergence de methodes de directions conjug´ es,

e e

Rev Fran¸ aise Informat Recherche Operationelle, 3e Ann´ e, 16 (1969), pp. 35“43.

c e

[216] B. T. Polyak, The conjugate gradient method in extremal problems, USSR Comp. Math.

and Math. Phys., 9 (1969), pp. 94“112.

[217] M. J. D. Powell, A FORTRAN Subroutine for Unconstrained Minimization, Requiring

First Derivatives of the Objective Function, Tech. Rep. AERE-R, 6469, Mathematics

Brance, A. E. R. E. Harwell, Berkshire, England, 1970.

, A hybrid method for nonlinear equations, in Numerical Methods for Nonlinear

[218]

Algebraic Equations, P. Rabinowitz, ed., Gordon and Breach, New York, 1970, pp. 87“

114.