This book is devoted to the systematic exposition of the contemporary theory of controlled Markov processes with discrete time parameter or in another termi nology multistage Markovian decision processes. We discuss the applications of this theory to various concrete problems. Particular attention is paid to mathe matical models of economic planning, taking account of stochastic factors. The authors strove to construct the exposition in such a way that a reader interested in the applications can get through the book with a minimal mathe matical apparatus. On the other hand, a mathematician will find, in the appropriate chapters, a rigorous theory of general control models, based on advanced measure theory, analytic set theory, measurable selection theorems, and so forth. We have abstained from the manner of presentation of many mathematical monographs, in which one presents immediately the most general situation and only then discusses simpler special cases and examples. Wishing to separate out difficulties, we introduce new concepts and ideas in the simplest setting, where they already begin to work. Thus, before considering control problems on an infinite time interval, we investigate in detail the case of the finite interval. Here we first study in detail models with finite state and action spaces-a case not requiring a departure from the realm of elementary mathematics, and at the same time illustrating the most important principles of the theory.

Controlled Markov Processes. A Series of Comprehensive Studies in Mathematics

Science & Math

Dynkin, E. B.

Mathematics

Springer

An investigation of the logical foundations of the theory behind Markov random processes, this text explores subprocesses, transition functions, and conditions for boundedness and continuity. Rather than focusing on probability measures individually, the work explores connections between functions. An elementary grasp of the theory of Markov processes is assumed. Starting with a brief survey of relevant concepts and theorems from measure theory, the text investigates operations that permit an inspection of the class of Markov processes corresponding to a given transition function. It advances to the more complicated operations of generating a subprocess, followed by examinations of the construction of Markov processes with given transition functions, the concept of a strictly "Markov process," and the conditions required for boundedness and continuity of a Markov process. Addenda, notes, references, and indexes supplement the text.

Theory of Markov Processes (Dover Books on Mathematics)

Dover `

Combining three books into a single volume, this text comprises Multicolor Problems, dealing with several of the classical map-coloring problems; Problems in the Theory of Numbers, an elementary introduction to algebraic number theory; and Random Walks, addressing basic problems in probability theory. The book's primary aim is not so much to impart new information as to teach an active, creative attitude toward mathematics. The sole prerequisites are high-school algebra and (for Multicolor Problems) a familiarity with the methods of mathematical induction. The book is designed for the reader's active participation. The problems are carefully integrated into the text and should be solved in order. Although they are basic, they are by no means elementary. Some sequences of problems are geared toward the mastery of a new method, rather than a definitive result, and others are practice exercises, designed to introduce new concepts. Complete solutions appear at the end.

Mathematical Conversations: Multicolor Problems, Problems in the Theory of Numbers, and Random Walks (Dover Books on Mathematics)

MATHEMATICS

Uspenskii, V. A.

Dover Publications

The theory of Markov Processes has become a powerful tool in partial differential equations and potential theory with important applications to physics. Professor Dynkin has made many profound contributions to the subject and in this volume are collected several of his most important expository and survey articles. The content of these articles has not been covered in any monograph as yet. This account is accessible to graduate students in mathematics and operations research and will be welcomed by all those interested in stochastic processes and their applications.

Markov Processes and Related Problems of Analysis (London Mathematical Society Lecture Note Series, Series Number 54)

Cambridge University Press

This book is devoted to the systematic exposition of the contemporary theory of controlled Markov processes with discrete time parameter or in another termi nology multistage Markovian decision processes. We discuss the applications of this theory to various concrete problems. Particular attention is paid to mathe matical models of economic planning, taking account of stochastic factors. The authors strove to construct the exposition in such a way that a reader interested in the applications can get through the book with a minimal mathe matical apparatus. On the other hand, a mathematician will find, in the appropriate chapters, a rigorous theory of general control models, based on advanced measure theory, analytic set theory, measurable selection theorems, and so forth. We have abstained from the manner of presentation of many mathematical monographs, in which one presents immediately the most general situation and only then discusses simpler special cases and examples. Wishing to separate out difficulties, we introduce new concepts and ideas in the simplest setting, where they already begin to work. Thus, before considering control problems on an infinite time interval, we investigate in detail the case of the finite interval. Here we first study in detail models with finite state and action spaces-a case not requiring a departure from the realm of elementary mathematics, and at the same time illustrating the most important principles of the theory.

Controlled Markov Processes (Grundlehren der mathematischen Wissenschaften)

Yushkevich, A. A.

The modem theory of Markov processes has its origins in the studies of A. A. MARKOV (1906-1907) on sequences of experiments "connected in a chain" and in the attempts to describe mathematically the physical phenomenon known as Brownian motion (L. BACHELlER 1900, A. EIN­ STEIN 1905). The first correct mathematical construction of a Markov process with continuous trajectories was given by N. WIENER in 1923. (This process is often called the Wiener process.) The general theory of Markov processes was developed in the 1930's and 1940's by A. N. KOL­ MOGOROV, W. FELLER, W. DOEBLlN, P. LEVY, J. L. DOOB, and others. During the past ten years the theory of Markov processes has entered a new period of intensive development. The methods of the theory of semigroups of linear operators made possible further progress in the classification of Markov processes by their infinitesimal characteristics. The broad classes of Markov processes with continuous trajectories be­ came the main object of study. The connections between Markov pro­ cesses and classical analysis were further developed. It has become possible not only to apply the results and methods of analysis to the problems of probability theory, but also to investigate analytic problems using probabilistic methods. Remarkable new connections between Markov processes and potential theory were revealed. The foundations of the theory were reviewed critically: the new concept of strong Markov process acquired for the whole theory of Markov processes great importance.

Markov Processes: Volume I (Grundlehren der mathematischen Wissenschaften, 121/122)

Crafts, Hobbies & Home

Gardening & Landscape Design

Markov Processes: Volume II (Grundlehren der mathematischen Wissenschaften)

The Dynkin Festschrift: Markov Processes and Their Applications

Birkhauser

See the best price to sell, buy, or rent books by Dynkin, E. B.. BookScouter helps to compare book prices from 25+ online bookstores and 30+ buyback vendors with a single search.

Books by Dynkin, E. B.