This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov decision processes or Markov dynamic programs. Starting in the mid-1950swith Richard Bellman, many contributions to CMP's have been made, and applications to engineering, statistics and operations research, among other areas, have also been developed. The purpose of this book is to present some recent developments on the theory of adaptive CMP's, i. e. , CMP's that depend on unknown parameters. Thus at each decision time, the controller or decision-maker must estimate the true parameter values, and then adapt the control actions to the estimated values. We do not intend to describe all aspects of stochastic adaptive control; rather, the selection of material reflects our own research interests. The prerequisite for this book is a knowledgeof real analysis and prob ability theory at the level of, say, Ash (1972) or Royden (1968), but no previous knowledge of control or decision processes is required. The pre sentation, on the other hand, is meant to beself-contained,in the sensethat whenever a result from analysisor probability is used, it is usually stated in full and references are supplied for further discussion, if necessary. Several appendices are provided for this purpose. The material is divided into six chapters. Chapter 1 contains the basic definitions about the stochastic control problems we are interested in; a brief description of some applications is also provided.

Adaptive Markov Control Processes (Applied Mathematical Sciences, 79)

Science & Math

Hernández-Lerma, Onésimo

Mathematics

Springer

Devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes, the text is mainly confined to MCPs with Borel state and control spaces. Although the book follows on from the author's earlier work, an important feature of this volume is that it is self-contained and can thus be read independently of the first. The control model studied is sufficiently general to include virtually all the usual discrete-time stochastic control models that appear in applications to engineering, economics, mathematical population processes, operations research, and management science.

Further Topics on Discrete-Time Markov Control Processes (Stochastic Modelling and Applied Probability, 42)

Lasserre, Jean B.

Evolutionary Games and the Replicator Dynamics (Elements in Evolutionary Economics)

This Element introduces the replicator dynamics for symmetric and asymmetric games where the strategy sets are metric spaces. Under this hypothesis the replicator dynamics evolves in a Banach space of finite signed measures. The authors provide a general framework to study the stability of the replicator dynamics for evolutionary games in this Banach space. This allows them to establish a relation between Nash equilibria and the stability of the replicator for normal a form games applicable to oligopoly models, theory of international trade, public good models, the tragedy of commons, and War of attrition game among others. They also provide conditions to approximate the replicator dynamics on a space of measures by means of a finite-dimensional dynamical system and a sequence of measure-valued Markov processes.

Business & Money

Mendoza-Palacios, Saul

Economics

Cambridge University Press

This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov decision processes or Markov dynamic programs. Starting in the mid-1950swith Richard Bellman, many contributions to CMP's have been made, and applications to engineering, statistics and operations research, among other areas, have also been developed. The purpose of this book is to present some recent developments on the theory of adaptive CMP's, i. e. , CMP's that depend on unknown parameters. Thus at each decision time, the controller or decision-maker must estimate the true parameter values, and then adapt the control actions to the estimated values. We do not intend to describe all aspects of stochastic adaptive control; rather, the selection of material reflects our own research interests. The prerequisite for this book is a knowledgeof real analysis and prob ability theory at the level of, say, Ash (1972) or Royden (1968), but no previous knowledge of control or decision processes is required. The pre sentation, on the other hand, is meant to beself-contained,in the sensethat whenever a result from analysisor probability is used, it is usually stated in full and references are supplied for further discussion, if necessary. Several appendices are provided for this purpose. The material is divided into six chapters. Chapter 1 contains the basic definitions about the stochastic control problems we are interested in; a brief description of some applications is also provided.

Adaptive Markov Control Processes (Applied Mathematical Sciences)

Devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes, the text is mainly confined to MCPs with Borel state and control spaces. Although the book follows on from the author's earlier work, an important feature of this volume is that it is self-contained and can thus be read independently of the first.The control model studied is sufficiently general to include virtually all the usual discrete-time stochastic control models that appear in applications to engineering, economics, mathematical population processes, operations research, and management science.

Further Topics on Discrete-Time Markov Control Processes (Stochastic Modelling and Applied Probability)

This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re source management, (control of) epidemics, etc. However, most of the lit erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.

Discrete-Time Markov Control Processes: Basic Optimality Criteria (Stochastic Modelling and Applied Probability)

Crafts, Hobbies & Home

Gardening & Landscape Design

This book concerns continuous-time controlled Markov chains and Markov games. The former, which are also known as continuous-time Markov decision processes, form a class of stochastic control problems in which a single decision-maker has a wish to optimize a given objective function. In contrast, there are two or more decision-makers (or players, or controllers) trying to optimize its own objective function in a Markov game. Both decision-making processes appear in a large number of applications in economics, operations research, engineering, and computer science — among other areas.The main features of the control and game models studied in the book are the continuous time variable, the denumerable state space, and that the control (or action) sets are Borel spaces. Moreover, the transition and reward rates of the dynamical system may be unbounded. The authors are interested in some aspects of controlled Markov chains and Markov games such as characterizing the optimal reward functions, and determining optimal policies for each of the optimality criteria studied here. The main focus is on advanced optimality criteria (such as, bias, variance, sensitive discount, and Blackwell optimality), though they also deal with the basic optimality criteria (discounted and average reward).A particular emphasis is made on the application of the results presented in this book. One of the main concerns is to propose assumptions on the control and game models that are easily verifiable (and verified) in practice. Moreover, algorithmic and computational issues are also analyzed. In particular, the authors propose approximation results that allow precise numerical approximations of the solution to some problems of practical interest. Applications to population models and epidemic processes are also shown.

SELECTED TOPICS ON CONTINUOUS-TIME CONTROLLED MARKOV CHAINS AND MARKOV GAMES (ICP Advanced Texts in Mathematics)

Prieto-Rumeau, Tomas

IMPERIAL COLLEGE PRESS

An Introduction to Optimal Control Theory: The Dynamic Programming Approach (Texts in Applied Mathematics, 76)

This book introduces optimal control problems for large families of deterministic and stochastic systems with discrete or continuous time parameter. These families include most of the systems studied in many disciplines, including Economics, Engineering, Operations Research, and Management Science, among many others. The main objective is to give a concise, systematic, and reasonably self contained presentation of some key topics in optimal control theory. To this end, most of the analyses are based on the dynamic programming (DP) technique. This technique is applicable to almost all control problems that appear in theory and applications. They include, for instance, finite and infinite horizon control problems in which the underlying dynamic system follows either a deterministic or stochastic difference or differential equation. In the infinite horizon case, it also uses DP to study undiscounted problems, such as the ergodic or long-run average cost.\nAfter a general introduction to control problems, the book covers the topic dividing into four parts with different dynamical systems: control of discrete-time deterministic systems, discrete-time stochastic systems, ordinary differential equations, and finally a general continuous-time MCP with applications for stochastic differential equations. 
The first and second part should be accessible to undergraduate students with some knowledge of elementary calculus, linear algebra, and some concepts from probability theory (random variables, expectations, and so forth). Whereas the third and fourth part would be appropriate for advanced undergraduates or graduate students who have a working knowledge of mathematical analysis (derivatives, integrals, ...) and stochastic processes.

Computers & Technology

Laura-Guarachi, Leonardo R.

González-Sánchez, David

Programming

This book is about discrete-time, time-homogeneous, Markov chains (Mes) and their ergodic behavior. To this end, most of the material is in fact about stable Mes, by which we mean Mes that admit an invariant probability measure. To state this more precisely and give an overview of the questions we shall be dealing with, we will first introduce some notation and terminology. Let (X,B) be a measurable space, and consider a X-valued Markov chain ~. = {~k' k = 0, 1, ... } with transition probability function (t.pJ.) P(x, B), i.e., P(x, B) := Prob (~k+1 E B I ~k = x) for each x E X, B E B, and k = 0,1, .... The Me ~. is said to be stable if there exists a probability measure (p.m.) /.l on B such that (*) VB EB. /.l(B) = Ix /.l(dx) P(x, B) If (*) holds then /.l is called an invariant p.m. for the Me ~. (or the t.p.f. P).

Markov Chains and Invariant Probabilities (Progress in Mathematics)

Management & Leadership

Birkhauser

See the best price to sell, buy, or rent books by Hernández-Lerma, Onésimo. BookScouter helps to compare book prices from 25+ online bookstores and 30+ buyback vendors with a single search.

Books by Hernández-Lerma, Onésimo

See the best price to sell, buy, or rent books by Hernández-Lerma, Onésimo. BookScouter helps to compare book prices from 25+ online bookstores and 30+ buyback vendors with a single search.

Adaptive Markov Control Processes (Applied Mathematical Sciences, 79)

Further Topics on Discrete-Time Markov Control Processes (Stochastic Modelling and Applied Probability, 42)

Evolutionary Games and the Replicator Dynamics (Elements in Evolutionary Economics)

Adaptive Markov Control Processes (Applied Mathematical Sciences)

Further Topics on Discrete-Time Markov Control Processes (Stochastic Modelling and Applied Probability)

Discrete-Time Markov Control Processes: Basic Optimality Criteria (Stochastic Modelling and Applied Probability)

SELECTED TOPICS ON CONTINUOUS-TIME CONTROLLED MARKOV CHAINS AND MARKOV GAMES (ICP Advanced Texts in Mathematics)

An Introduction to Optimal Control Theory: The Dynamic Programming Approach (Texts in Applied Mathematics, 76)

An Introduction to Optimal Control Theory: The Dynamic Programming Approach (Texts in Applied Mathematics, 76)

Markov Chains and Invariant Probabilities (Progress in Mathematics)