This article includes a list of general references, but it remains largely unverified because it lacks sufficient corresponding inline citations. (June 2020) (Learn how and when to remove this template message)
Shape optimization is part of the field of optimal control theory. The typical problem is to find the shape which is optimal in that it minimizes a certain cost functional while satisfying given constraints. In many cases, the functional being solved depends on the solution of a given partial differential equation defined on the variable domain.
Topology optimization is, in addition, concerned with the number of connected components/boundaries belonging to the domain. Such methods are needed since typically shape optimization methods work in a subset of allowable shapes which have fixed topological properties, such as having a fixed number of holes in them. Topological optimization techniques can then help work around the limitations of pure shape optimization.
possibly subject to a constraint of the form
Usually we are interested in sets which are Lipschitz or C1 boundary and consist of finitely many components, which is a way of saying that we would like to find a rather pleasing shape as a solution, not some jumble of rough bits and pieces. Sometimes additional constraints need to be imposed to that end to ensure well-posedness of the problem and uniqueness of the solution.
Shape optimization is an infinite-dimensional optimization problem. Furthermore, the space of allowable shapes over which the optimization is performed does not admit a vector space structure, making application of traditional optimization methods more difficult.
- Among all three-dimensional shapes of given volume, find the one which has minimal surface area. Here:
- Find the shape of an airplane wing which minimizes drag. Here the constraints could be the wing strength, or the wing dimensions.
- Find the shape of various mechanical structures, which can resist a given stress while having a minimal mass/volume.
- Given a known three-dimensional object with a fixed radiation source inside, deduce the shape and size of the source based on measurements done on part of the boundary of the object. A formulation of this inverse problem using least-squares fit leads to a shape optimization problem.
Shape optimization problems are usually solved numerically, by using iterative methods. That is, one starts with an initial guess for a shape, and then gradually evolves it, until it morphs into the optimal shape.
Keeping track of the shape
To solve a shape optimization problem, one needs to find a way to represent a shape in the computer memory, and follow its evolution. Several approaches are usually used.
One approach is to follow the boundary of the shape. For that, one can sample the shape boundary in a relatively dense and uniform manner, that is, to consider enough points to get a sufficiently accurate outline of the shape. Then, one can evolve the shape by gradually moving the boundary points. This is called the Lagrangian approach.
Another approach is to consider a function defined on a rectangular box around the shape, which is positive inside of the shape, zero on the boundary of the shape, and negative outside of the shape. One can then evolve this function instead of the shape itself. One can consider a rectangular grid on the box and sample the function at the grid points. As the shape evolves, the grid points do not change; only the function values at the grid points change. This approach, of using a fixed grid, is called the Eulerian approach. The idea of using a function to represent the shape is at the basis of the level-set method.
A third approach is to think of the shape evolution as of a flow problem. That is, one can imagine that the shape is made of a plastic material gradually deforming such that any point inside or on the boundary of the shape can be always traced back to a point of the original shape in a one-to-one fashion. Mathematically, if is the initial shape, and is the shape at time t, one considers the diffeomorphisms
The idea is again that shapes are difficult entities to be dealt with directly, so manipulate them by means of a function.
Iterative methods using shape gradients
Consider a smooth velocity field and the family of transformations of the initial domain under the velocity field :
Then the Gâteaux or shape derivative of at with respect to the shape is the limit of
if this limit exists. If in addition the derivative is linear with respect to , there is a unique element of and
where is called the shape gradient. This gives a natural idea of gradient descent, where the boundary is evolved in the direction of negative shape gradient in order to reduce the value of the cost functional. Higher order derivatives can be similarly defined, leading to Newtonlike methods.
Typically, gradient descent is preferred, even if requires a large number of iterations, because, it can be hard to compute the second-order derivative (that is, the Hessian) of the objective functional .
If the shape optimization problem has constraints, that is, the functional is present, one has to find ways to convert the constrained problem into an unconstrained one. Sometimes ideas based on Lagrange multipliers can work.
Shape optimization can be faced using standard optimization methods if a parametrization of the geometry is defined. Such parametrization is very important in CAE field where goal functions are usually complex functions evaluated using numerical models (CFD, FEA,...). A convenient approach, suitable for a wide class of problems, consists in the parametrization of the CAD model coupled with a full automation of all the process required for function evaluation (meshing, solving and result processing). Mesh morphing is a valid choice for complex problems that resolves typical issues associated with re-meshing such as discontinuities in the computed objective and constraint functions . In this case the parametrization is defined after the meshing stage acting directly on the numerical model used for calculation that is changed using mesh updating methods. There are several algorithms available for mesh morphing (deforming volumes, pseudosolids, radial basis functions). The selection of the parametrization approach depends mainly on the size of the problem: the CAD approach is preferred for small-to-medium sized models whilst the mesh morphing approach is the best (and sometimes the only feasible one) for large and very large models. The multi-objective Pareto optimization (NSGA II) could be utilized as a powerful approach for shape optimization. In this regard, the Pareto optimization approach displays useful advantages in design method such as the effect of area constraint that other multi-objective optimization cannot declare it. The approach of using a penalty function is an effective technique which could be used in the first stage of optimization. In this method the constrained shape design problem is adapted to an unconstrained problem with utilizing the constraints in the objective function as a penalty factor. Most of the time penalty factor is dependent to the amount of constraint variation rather than constraint number. The GA real-coded technique is applied in the present optimization problem. Therefore, the calculations are based on real value of variables. 
- Wilke, D.N.; Kok, S.; Groenwold, A.A. (2010) The application of gradient-only optimization methods for problems discretized using non-constant methods. Structural and Multidisciplinary Optimization, Vol. 40, 433-451.
- Talebitooti, R.; shojaeefard, M.H.; Yarmohammadisatri, Sadegh (2015). "Shape design optimization of cylindrical tank using b-spline curves". Computer & Fluids. 109: 100–112. doi:10.1016/j.compfluid.2014.12.004.
- Allaire, G. (2002) Shape optimization by the homogenization method. Applied Mathematical Sciences 146, Springer Verlag. ISBN 0-387-95298-5
- Ashok D. Belegundu, Tirupathi R. Chandrupatla. (2003) Optimization Concepts and applications in Engineering Prentice Hall. ISBN 0-13-031279-7.
- Bendsøe M. P.; Sigmund O. (2003) Topology Optimization: Theory, Methods and Applications. Springer. ISBN 3-540-42992-1.
- Burger, M.; Osher, S.L. (2005) A Survey on Level Set Methods for Inverse Problems and Optimal Design. European Journal of Applied Mathematics, vol.16 pp. 263–301.
- Delfour, M.C.; Zolesio, J.-P. (2001) Shapes and Geometries - Analysis, Differential Calculus, and Optimization. SIAM. ISBN 0-89871-489-3.
- Haslinger, J.; Mäkinen, R. (2003) Introduction to Shape Optimization: Theory, Approximation and Computation. Society for Industrial and Applied Mathematic. ISBN 0-89871-536-9.
- Laporte, E.; Le Tallec, P. (2003) Numerical Methods in Sensitivity Analysis and Shape Optimization. Birkhäuser. ISBN 0-8176-4322-2.
- Mohammadi, B.; Pironneau, O. (2001) Applied Shape Optimization for Fluids. Oxford University Press. ISBN 0-19-850743-7.
- Simon J. (1980) Differentiation with respect to the domain in boundary value problems. Numer. Fuct. Anal. and Optimiz., 2(7&8), 649-687 (1980).
- Optopo Group — Simulations and bibliography of the optopo group at Ecole Polytechnique (France). Homogenization method and level set method.