Mathematics - Optimization 2
Branches of Optimization Research #
Convex Optimization #
Convex optimization focuses on problems where the objective function and constraints are convex, ensuring a single global optimum. This field is foundational in machine learning, signal processing, and control systems due to its guaranteed convergence and efficient algorithms.
- Convex Optimization by Boyd and Vandenberghe - PDF
- Convex Optimization Theory by Dimitri P. Bertsekas - PDF
Discrete, Combinatorial, and Integer Optimization #
This branch deals with optimization problems involving discrete variables, such as integers or combinatorial structures, often encountered in scheduling, network design, and logistics. Bayesian optimization, a subset, is particularly useful for optimizing expensive black-box functions.
- Bayesian Optimization In Action by Quan Nguyen - Amazon
- Experimentation for Engineers by David Sweet - Amazon
Operations Research #
Operations research applies mathematical modeling and optimization to complex decision-making in logistics, supply chain, and resource allocation. It integrates techniques like linear programming, simulation, and heuristic methods to optimize real-world systems.
- Operations Research An Introduction by Hamdy A. Taha - Pearson
- Introduction to Operations Research by Frederick Hillier and Gerald Lieberman - McGraw Hill
- Julia Programming for Operations Research by Changhyun Kwon - PDF - code
- Mathematical Programming and Operations Research: Modeling, Algorithms, and Complexity. Examples in Python and Julia. Edited by Robert Hildebrand - PDF
- A First Course in Linear Optimization by Jon Lee - PDF
- Decomposition Techniques in Mathematical Programming by Conejo , Castillo , Mínguez , and García-Bertrand - Springer
- Algorithms for Optimization by Mykel J. Kochenderfer and Tim A. Wheeler - PDF
- Model Building in Mathematical Programming - Introductory modeling book by H. Paul Williams - Wiley
Meta-heuristics #
Meta-heuristics are high-level strategies for solving complex optimization problems where exact methods are computationally infeasible. They include nature-inspired algorithms like genetic algorithms and simulated annealing, widely used in engineering and data science.
- Metaheuristics by Patrick Siarry - Springer (open access)
- Essentials of Metaheuristics by Sean Luke - link
- Handbook of Metaheuristics by Michel Gendreau and Jean-Yves Potvin - Springer (open access)
- An Introduction to Metaheuristics for Optimization by Bastien Chopard , Marco Tomassini - Springer (open access)
- Metaheuristic and Evolutionary Computation: Algorithms and Applications by Hasmat Malik, Atif Iqbal, Puneet Joshi, Sanjay Agrawal, and Farhad Ilahi Bakhsh - Springer (open access)
- Clever Algorithms: Nature-Inspired Programming Recipes by Jason Brownlee - GitHub
- Metaheuristics: from design to implementation by El-Ghazali Talbi - Wiley
Dynamic Programming and Reinforcement Learning #
Dynamic programming and reinforcement learning address sequential decision-making problems, breaking them into subproblems or learning optimal policies through interaction with environments. These methods are critical in robotics, finance, and AI.
- Various tiltes on Dynamic Programming, Optimal Control and Reinforcement Learning by Dimitri Bertsekas. - List
- Reinforcement Learning: An Introduction (2nd Edition) by Richard Sutton and Andrew Barto - PDF
- Decision Making Under Uncertainty: Theory and Application by Mykel J. Kochenderfer - PDF
- Algorithms for Decision Making by Mykel J. Kochenderfer, Tim A. Wheeler, and Kyle H. Wray - PDF
Constraint Programming #
Constraint programming solves problems by defining constraints that must be satisfied, often used in scheduling, planning, and configuration tasks. It excels in problems with complex logical constraints and discrete variables.
- Handbook of Constraint Programming by Francesca Rossi, Peter van Beek and Toby Walsh - Amazon
- A Tutorial on Constraint Programming by Barbara M. Smith (University of Leeds) - PDF
Combinatorial Optimization #
Combinatorial optimization focuses on finding optimal solutions in discrete structures, such as graphs or sets, often using algorithms for problems like the traveling salesman or graph coloring, with applications in logistics and network design.
- Combinatorial Optimization: Algorithms and Complexity by by Christos H. Papadimitriou and Kenneth Steiglitz - Amazon
- Combinatorial Optimization: Theory and Algorithms by Bernhard Korte and Jens Vygen - Springer
- A First Course in Combinatorial Optimization by Jon Lee - Amazon
Stochastic Optimization and Control #
Stochastic optimization handles problems with uncertainty or randomness, using probabilistic models to optimize objectives. It is widely applied in machine learning, finance, and operations research for robust decision-making.
- Lectures on Stochastic Programming Modeling and Theory (SIAM) - by Shapiro, Dentcheva, and Ruszczynski - PDF
- Introductory Lectures on Stochastic Optimization by John C. Duchi - PDF