Abstract. We discuss the stability of "critical" or "equilibrium" shapes of a shape-dependent energy functional. We analyze a problem arising when looking at the positivity of the second derivative in order to prove that a critical shape is an optimal shape. Indeed, often when positivity -or coercivityholds, it does for a weaker norm than the norm for which the functional is twice differentiable and local optimality cannot be a priori deduced. We solve this problem for a particular but significant example. We prove "weak-coercivity" of the second derivative uniformly in a "strong" neighborhood of the equilibrium shape.Résumé. Nous nous intéressonsà la stabilité des formes critiques ou d'"équilibre" d'uneénergie dépendant de la forme. Dans le but de montrer qu'une forme critique est une forme optimale, nouś etudions la positivité de la dérivée seconde. En effet, quand elle a lieu, la coercivité n'est vraie que dans une norme plus faible que celle pour laquelle l'énergie est différentiable: l'optimalité locale ne peut donc pas enêtre déduite a priori. Nous résolvons cette difficulté dans un cas particulier mais néanmoins significatif. Nousétablissons de la "coercivité faible" de la dérivée seconde uniformément dans un voisinage "fort" de la forme d'équilibre.Mathematics Subject Classification. 49Q10, 49K40, 35J25.

We are interested in the question of stability in the field of shape optimization, with focus on the strategy using second order shape derivative. More precisely, we identify structural hypotheses on the hessian of the considered shape function, so that critical stable domains (i.e. such that the first order derivative vanishes and the second order one is positive) are local minima for smooth perturbations; as we are in an infinite dimensional framework, and that in most applications there is a norm-discrepancy phenomenon, this type of result require a lot of work. We show that these hypotheses are satisfied by classical functionals, involving the perimeter, the Dirichlet energy or the first Laplace-Dirichlet eigenvalue. We also explain how we can easily deal with constraints and/or invariance of the functionals. As an application, we retrieve or improve previous results from the existing literature, and provide new local stability results. We finally test the sharpness of our results by showing that the local minimality is in general not valid for non-smooth perturbations.2000MSC : 49K20, 49Q10.Keywords : isoperimetric inequalities, shape optimization, second order sensitivity, stability in shape optimization.One then uses (in the whole paper) the usual notion of Fréchet-differentiability: shape derivatives of J at Ω are the successive derivatives of J Ω at 0, when they exist. In particular, the first shape derivative is J ′ (Ω) := J ′ Ω (0), a continuous linear form on Θ (the shape gradient), and the second order shape derivative is J ′′ (Ω) := J ′′ Ω (0), a continuous symmetric bilinear form on Θ (the shape hessian). • Normal graphs:On the other hand, assuming that Ω is C 1 (and n = n ∂Ω is its outer unit normal vector) we can considerWith the last formula, we easily notice that Q 1 = 0. The sign of Q k can be obtained using [38, section 6.5 page 133] (done when d = 2, but as noticed in [31], valid for any d): indeed, their computations implywhich leads to ∀k ≥ 2, Q k ≥ k − 1. Therefore Theorem 3.2 applies, and we retrieve a Faber-Krahn quantitative inequality in a W 2,p -neighborhood of the ball. Examples with competitionIn this section, B is a ball, X = W 2,p (∂B) for p > d and we denote for η > 0 (see (18) for a definition of d X ):Combining Theorem 3.2 to the computations from Section 2.1, we easily obtain the following result:Proposition 5.1 There exists γ 0 ∈ (0, ∞) such that for every γ ∈ [−γ 0 , ∞), there exists η = η(γ) > 0 and c = c(γ) > 0 such that for every Ω ∈ V η ,Proof of Proposition 5.1: We show that we can apply Theorem 3.2 can be applied to Ω * = B and J ∈ {P + γE, P + γλ 1 , E + γλ 1 , λ 1 + γE)}.It is shown in Sections 3.2 and 4 that (P, E, λ 1 ) satisfy (C H s 2 ) and (IT H s 2 ,X ) for suitable values of s 2 , and with Lemmata 2.9 and 2.8 we easily check that the ball is a critical and strictly stable domain for J under volume constraint and up to translations, either if γ ≥ 0 or if γ < 0 is small enough.Corollary 5.2 With the same notations as in Proposition 5.1, we have, with η 0 = η(γ ...

The paper presents a theoretical study of an identification problem by shape optimization methods. The question is to detect an object immersed in a fluid. Here, the problem is modeled by the Stokes equations and treated as a nonlinear least-squares problem. We consider both the Dirichlet and Neumann boundary conditions. Firstly, we prove an identifiability result. Secondly, we prove the existence of the first-order shape derivatives of the state, we characterize them and deduce the gradient of the least-squares functional. Moreover, we study the stability of this setting. We prove the existence of the second-order shape derivatives and we give the expression of the shape Hessian. Finally, the compactness of the Riesz operator corresponding to this shape Hessian is shown and the ill-posedness of the identification problem follows. This explains the need of regularization to numerically solve this problem.