Why Does the Levenberg-Marquardt Method Optimize on the Sum of the Squares of f, Instead of Just f? Primary Software: Other NI Software>>HiQPrimary Software Version: 4.1 Primary Software Fixed Version: N/A Secondary Software: N/A
Problem: When I look at the optimize function in HiQ, I see that the quasi-Newton, conjugate gradient and Nelder-Mead methods all optimize f(x). Why does the Levenberg-Marquardt method optimize on the sum of f(x)^2? Solution: The difference in the methods is in what is getting optimized. The quasi-Newton, conjugate gradient, and Nelder-Mead methods all optimize "nonlinear single-valued functions." That is, you have one equation based on several variables to minimize, thus you minimize f(x). The Levenberge-Marquardt method optimizes "nonlinear multi-valued functions." That is, you have several equations based on the same set of variables and you want to minimize ALL of these equations at the same time. You are not necessarily looking for the minimum of each function, but the set of variables that gives the minimum of a combination of the functions. For example: Say that you have a factory and a certain production process has three variables: temperature (T), pressure (P), and flow rate (F). Based on these three inputs, you have 3 outputs: number of units (N), quality of units (Q), and errors (E). If I wanted to optimize just N, I would use either the quasi-Newton, conjugate, or Nelder-Mead method. But this may not give me the optimum value for Q or E. What if we want to optimize the whole process? We could just optimize on N, Q, or E individually, but this would give us three sets of variables. What we really want is to optimize the whole process for one set of variable, so we make one function that includes all three of these equations, f(x). There are several equations inside of the one function. When you optimize on this function, you look for the Euclidian Norm (square root of the sum of the squares). You try to minimize not just the output value of one function, but the minimum value of a combination of the outputs of all the functions. To do this, we use the square root (sqrt) of the sum of the squares, but this is the same as minimizing the sum of the squares. The values for T, P, and F that are found probably won't produce the best N, Q, or E values individually, but they will be the best combination of the three. Therefore, the minimum value (f-min) returned will be the smallest sum of the squared outputs of N, Q, and E at the T, P, and F found. T, P, and F will be returned in the x-min vector. To find the minimum values of N, Q, and E, you will need to plug T, P, and E back into their respective equations. Note: National Instruments does not support HiQ functionality on Mac OS in LabVIEW 7.0 and will no longer support HiQ functionality on all platforms after LabVIEW 7.0. If your application uses HiQ VIs, consider replacing them with the Analyze and Mathematics VIs. Refer to the LabVIEW Help for information about using the Analyze and Mathematics VIs. You access the LabVIEW Help by selecting Help»VI, Function, & How-To Help. Related Links: HiQ Support Attachments:
Report Date: 02/16/2001 Last Updated: 05/28/2004 Document ID: 26F87UPQ |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
