Finite

Newton-Raphson method is one of the methods used to numerically find the roots of a nonlinear equation. It is characterized by its rapid convergence of approximations to the actual solution, although its convergence speed and whether it converges at all can depend on the starting point. In this post, I'll geometrically derive the Newton-Raphson formula using Taylor series expansion. Firstly, the..

In this post, I'll solve an example using the the SQP algorithm described in the previous post. Starting point is $(1,1)$, ${R_0}=10$, $\gamma = 0.5$, and $\varepsilon = 0.001$. The initial iteration of the SQP algorithm is identical to the CSD algorithm. The result of the first iteration is as follows. At ${x^2}$, the gradients for the cost function and constraints are calculated as follows. Ca..

When defining the subproblem for determining the search direction in nonlinear problems, algorithms that only use linear approximations of cost and constraint functions may have the drawback of slow convergence rates. Incorporating second-order information about the problem functions into the solution process can improve this. The Sequential Quadratic Programming (SQP) method, which I'll discuss..

In the previous post, I saw the equation for projecting the design variable vector onto the constraint hyperplane. This approach is part of what's known as projection methods, like the projected conjugate gradient method, projected BFGS method, etc. The basic idea of projection methods is to fix the variables at the boundary to their boundary values in each iteration, as long as they satisfy the..

Constrained optimization problems can be transformed into unconstrained problems using penalty or Lagrangian approaches. In this case, explicitly handling boundary constraints for variables in numerical optimization algorithms can be more efficient. A constrained optimization problem can be defined as follows. For the ${i^{th}}$ variable, ${x_{iL}}$ and ${x_{iU}}$ represent the lower and upper b..

One of the numerical methods for constrained optimization design, the CSD algorithm, calculates step size using the golden section search. While this method is quite effective among interval reduction techniques, it's inefficient for complex engineering applications due to the need for too many function evaluations. Therefore, most practical algorithm implementations use an inexact line search t..

Numerical algorithms that employ a potential constraint strategy need to prove convergence. Here, the potential set strategy has been integrated into the Constrained Steepest Descent (CSD) algorithm, which has been proven to converge to a local minimum at every point. Therefore, it has been proven that using the potential set strategy converges to a local minimum from various points. In this pos..

In most problems, it's important to note that only some of the inequality constraints may be active at the minimum point. Moreover, the part of these activated constraints isn't known beforehand and must be determined during the problem-solving process. This post introduces the concept of potentially active constraints at the minimum point, a concept that can be integrated into numerical algorit..

Based on what was written in the previous post(124_Linearization), I plan to explain the process of transforming a problem into a linearized subproblem using an example. In the given problem, linearize the cost and constraint function based on the point ${x^{(0)}} = \left( {1,1} \right)$. By substituting ${x^{(0)}}$ into the cost and constraint functions and calculating, the results are as follo..

Most numerical optimization methods calculate design changes by solving a subproblem obtained through the linear Taylor expansion of cost and constraint functions at every iteration. This idea of approximating or linearizing the subproblem is crucial in numerical optimization methods and needs to be understood. Every search method starts with an initial estimate of the design and iteratively imp..