We consider zero-sum stochastic games with Borel state spaces satisfying a generalized geometric ergodicity condition. The main objective of this paper is to establish a minimax theorem for a class of ergodic stochastic games with the Feller transition ...

We consider an infinite horizon discounted cost minimization problem for a one-dimensional stochastic differential equation model. The available control is an added bounded variation process. The cost structure involves a running cost function and a ...

The value function for the average cost control of a class of partially observed Markov chains is derived as the "vanishing discount limit," in a suitable sense, of the value functions for the corresponding discounted cost problems. The limiting ...

In this paper, we study a form of stability for a general family of nondiffusion Markov processes known in the literature as piecewise-deterministic Markov process (PDMP). By stability here we mean the existence of an invariant probability measure for ...

We investigate the convergence of the solution set for singularly perturbed differential inclusions using an averaging technique. The limits of the fast solutions are considered as generalized solutions (i.e., Radon probability measures) of the ...

In this paper we provide a relaxation result for control systems under both equality and inequality constraints involving the state and the control. In particular, we show that the Mangasarian--Fromowitz constraint qualification allows us to rewrite ...

Three distinct controlled ergodic Markov models are considered here. The models are a discrete time controlled Markov process with complete observations, a controlled diffusion process with complete observations, and a discrete time controlled Markov ...

Given a solution of a controlled martingale problem it is shown under general conditions that there exists a solution having Markov controls which has the same cost as the original solution. This result is then used to show that the original stochastic ...

In this paper we use infinite linear programming to study Markov control processes in Borel spaces and the average cost criterion in the "unichain" and "multichain" cases. Under appropriate assumptions we show that in both cases the associated linear ...

Controlled coupled slow and fast motions are examined in a singular perturbations setting. The objective is to minimize a cost functional that takes into account both the fast motion, supposing, say, tracking a fast target, and the slow dynamics. A ...

In this paper we consider robust and risk-sensitive control of discrete time finite state systems on an infinite horizon. The solution of the state feedback robust control problem is characterized in terms of the value of an average cost dynamic game. ...

In this paper we consider an optimal control problem for a class of systems governed by nonlinear evolution equations containing a nonlinear monotone operator and a nonmonotone operator with uncertain parameters. We prove existence of solutions and ...

Weighted averages of Kiefer--Wolfowitz-type procedures, which are driven by larger step lengths than usual, can achieve the optimal rate of convergence. A priori knowledge of a lower bound on the smallest eigenvalue of the Hessian matrix is avoided. The ...

Stochastic control problems on a finite horizon with exponential cost criteria are considered. By taking a kind of singular limit a Hamilton--Jacobi--Isaacs equation is obtained. Its solution is characterized as the lower value function of a ...

For multivariable, input-output systems that are represented as rational, transfer-function matrices, the most frequently used measures of relative stability are gain based. However, there are a number of important physical applications where the phase ...

Most physical processes exhibit transport delay in the measured output, and it is well known that this can have disastrous effects on system stability and performance if it is not accounted for. In this paper, we give necessary and sufficient conditions ...

The concept of estimation algebra introduced independently by Brockett and Mitter has been playing a fundamental role in the investigation of finite-dimensional nonlinear filters. Mitter conjectured that the observation terms $h_i(x)$ are polynomials of ...

Optimization problems involving an infinite number of constraints are considered. This paper presents a general stochastic outer approximations method which incorporates mechanisms for active search of relevant constraints and for dropping of irrelevant ...

We study a stochastic control problem for the optimization of observations in a partially observable stochastic system. Using a method of discontinuous time transformation, we associate with the original problem with unbounded controls a problem that ...

In this paper we give a new proof of the existence of Puiseux series expansion of the limit discount equation of finite state stochastic games. Unlike the original proof, due to Bewley and Kohlberg [ Math. Oper. Res., 3 (1976), pp. 197--208], our proof ...