Constrained Markov Decision Processes
Regular price
€235.60
14 days return policy
Shipping & Delivery
A01=Eitan Altman
approximation
Author_Eitan Altman
Behavioral Policies
Category=PBT
Category=PBWL
Closed Convex Hull
CMDP
Constrained Control Problem
constrained stochastic process optimization
Controlled Markov Chains
Cost criteria
Countable State Space
Discounted Cost Criterion
Discounted Cost Problem
Dual Linear Program
Dual LP.
Duality Gap
dynamic programming methods
eq_isMigrated=1
eq_isMigrated=2
eq_nobargain
Finite Horizon
Infinite State Space
Initial Distribution
linear programming optimization
Markov Policy
MDP
Occupation measure
occupation measure analysis
Optimal Policies
Optimal Stationary Policies
policy randomization strategies
queueing systems modeling
State Action Pair
State truncation
stochastic control theory
Superharmonic Function
The dual LP
The Dual Program
The Lagrangian
the methodology
The model
Truncated Problems
Unbounded Cost
Uniform Integrability
Vice Versa
Product details
- ISBN 9780849303821
- Weight: 520g
- Dimensions: 152 x 229mm
- Publication Date: 30 Mar 1999
- Publisher: Taylor & Francis Ltd
- Publication City/Country: GB
- Product Form: Hardback
Secure
checkout
Fast Shipping
Easy returns
This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction.
The book is then divided into three sections that build upon each other.
The first part explains the theory for the finite state space. The author characterizes the set of achievable expected occupation measures as well as performance vectors, and identifies simple classes of policies among which optimal policies exist. This allows the reduction of the original dynamic into a linear program. A Lagranian approach is then used to derive the dual linear program using dynamic programming techniques.
In the second part, these results are extended to the infinite state space and action spaces. The author provides two frameworks: the case where costs are bounded below and the contracting framework.
The third part builds upon the results of the first two parts and examines asymptotical results of the convergence of both the value and the policies in the time horizon and in the discount factor. Finally, several state truncation algorithms that enable the approximation of the solution of the original control problem via finite linear programs are given.
Altman, Eitan
