Deterministic Global Optimization with Artificial Neural Networks Embedded

Schweidtmann, Artur M.; Mitsos, Alexander

doi:10.1007/s10957-018-1396-0

Cited by 172 publications

(130 citation statements)

References 70 publications

Supporting

Mentioning

125

Contrasting

Unclassified

Order By: Relevance

“…We are thus compelled to search for a good tradeoff between model depth and optimization efficiency. The same observation was reported in [55,36] for global optimization of DNNs, supporting the claim that deeper networks are more challenging to optimize. To conclude, we observe, as others have before us and as theory predicts, that the feasibility of using the MILP formulation quickly fades with increasing network sizes.…”

Section: Discussionsupporting

confidence: 81%

ReLU networks as surrogate models in mixed-integer linear programs

Grimstad

Andersson

2019

Computers & Chemical Engineering

117

View full text Add to dashboard Cite

We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the big-M method. The efficiency of the formulation hinges on the tightness of the bounds defined by the big-M values. When ReLU networks are embedded in a larger optimization problem, the presence of output bounds can be exploited in bound tightening. To this end, we devise and study several bound tightening procedures that consider both input and output bounds. Our numerical results show that bound tightening may reduce solution times considerably, and that small-sized ReLU networks are suitable as surrogate models in mixed-integer linear programs.

show abstract

Section: Discussionsupporting

confidence: 81%

ReLU networks as surrogate models in mixed-integer linear programs

Grimstad

Andersson

2019

Computers & Chemical Engineering

117

View full text Add to dashboard Cite

show abstract

“…We impose the annual operation cost C operation as a constraint and minimize the permeate concentration. [55] which is favorable for flowsheet problems [38,56] and optimization problems with ANNs embedded [20,21,35,36,57]. We also considered a full-space formulation with the solver BARON (version 18.5.8 using default options) [51] in GAMS [58] (version 25.1.1).…”

Section: Numerical Optimization Approachmentioning

confidence: 99%

“…For layer-by-layer (LbL) nanofiltration membranes, we demonstrated in our recent work that an artificial neural network (ANN) approach can predict ion separation properties and pure water flux directly based on synthesis protocols, i.e., the fabrication parameters of the membrane [20]. Contrary to conventional screening procedures or educated guess experimental design, we developed a deterministic global optimization method [20,21] that is applied in this work to enable optimal tailoring of synthesis protocols towards desired membrane retention and permeability in process optimization. Moreover, this framework is versatile enough to compute the resulting trade-off boundary (Pareto front) between permeability and retention.…”

Section: Introductionmentioning

confidence: 99%

“…Both the operational variables and the membrane selection variable are degrees of freedom optimized by the deterministic global optimization solver MAiNGO [37]. This algorithm is well-suited for optimization problems with ANNs embedded [21] and flowsheet optimization problems [38]. Second, this conventional optimization strategy is extended by the simultaneous design of the LbL membrane modules (c.f., Figure 1 Novel hybrid membrane model, index (SD)) with the synthesis protocol of the membrane being a new and important degree of freedom of the optimization problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Simultaneous rational design of ion separation membranes and processes

Rall

Schweidtmann

Aumeier

et al. 2020

Journal of Membrane Science

Self Cite

View full text Add to dashboard Cite

Economically viable water treatment process plants for drinking water purification are a prerequisite for sustainable supply of safe drinking water in the future. However, modern membrane process development experiences a disconnect in this domain: the synthesis of the membrane and the design of the process are decoupled. We propose an optimization strategy to simultaneously design the performance of layer-by-layer nanofiltration membrane modules and the separation process. This approach achieves overall optimal performance by extending the search space and thus exploiting synergies. Better separation performances at a lower cost as compared to conventional optimization strategies can be achieved. The key feature of this optimization framework is the integration of artificial neural networks. This machine-learning technique describes membrane performance as a function of its synthesis protocol. We optimize the design problem rigorously by a deterministic global nonlinear optimization method. Thus, this framework yields membrane synthesis protocols and membrane processes that are optimally tailored to the desired separation task. In a showcase, the simultaneous membrane synthesis and process optimization design achieve immediately favorable results with lower impurities at comparable costs. The process investment and operation costs are compared to a state of the art commercially available membrane for nanofiltration.

show abstract

“…Together with other classes of approximators, both these kinds of networks are applied extensively by the so-called Extended Ritz Method [43], [44], [45], allowing to get suboptimal solutions to functional optimization problems with guarantees on their performance. Likewise Gaussian RBF networks, also feedforward neural networks with sigmoidal computational units can be applied to perform surrogate optimization [46]; d) still in order to use a coarser grid and allow at the same time a precise evaluation of the band gap amplitude, machine-learning methods could be applied for dispersion curve identification in the presence of curve intersections [47].…”

Section: Conclusion and Further Research Directionsmentioning

confidence: 99%

Machine-Learning Techniques for the Optimal Design of Acoustic Metamaterials

Bacigalupo

Gnecco

Lepidi

et al. 2019

J Optim Theory Appl

View full text Add to dashboard Cite

Recently, an increasing research effort has been dedicated to analyse the transmission and dispersion properties of periodic acoustic metamaterials, characterized by the presence of local resonators. Within this context, particular attention has been paid to the optimization of the amplitudes and center frequencies of selected stop and pass bands inside the Floquet-Bloch spectra of the acoustic metamaterials featured by a chiral or antichiral microstructure. Novel functional applications of such research are expected in the optimal parametric design of smart tunable mechanical filters and directional waveguides. The present paper deals with the maximization of the amplitude of low-frequency band gaps, by proposing suitable numerical techniques to solve the associated optimization problems. Specifically, the feasibility and effectiveness of Radial Basis Function networks and Quasi-Monte Carlo methods for the interpolation of the objective functions of such optimization problems are discussed, and their numerical application to a specific acoustic metamaterial with tetrachiral microstructure is presented. The discussion is motivated theoretically by the high computational effort often needed for an exact evaluation of the objective functions arising in band gap optimization problems, when iterative algorithms are used for their approximate solution. By replacing such functions with suitable surrogate objective functions constructed applying machine-learning techniques, well-performing suboptimal solutions can be obtained with a smaller computational effort. Numerical results demonstrate the effective potential of the proposed approach. Current directions of research involving the use of additional machine-learning techniques are also presented.

show abstract

Deterministic Global Optimization with Artificial Neural Networks Embedded

Cited by 172 publications

References 70 publications

ReLU networks as surrogate models in mixed-integer linear programs

ReLU networks as surrogate models in mixed-integer linear programs

Simultaneous rational design of ion separation membranes and processes

Machine-Learning Techniques for the Optimal Design of Acoustic Metamaterials

Contact Info

Product

Resources

About