A First Look At Stochastic Processes

This textbook introduces the theory of stochastic processes, that is, randomness which proceeds in time. Using concrete examples like repeated gambling and jumping frogs, it presents fundamental mathematical results through simple, clear, logical theorems and examples. It covers in detail such essential material as Markov chain recurrence criteria, the Markov chain convergence theorem, and optional stopping theorems for martingales. The final chapter provides a brief introduction to Brownian motion, Markov processes in continuous time and space, Poisson processes, and renewal theory.Interspersed throughout are applications to such topics as gambler's ruin probabilities, random walks on graphs, sequence waiting times, branching processes, stock option pricing, and Markov Chain Monte Carlo (MCMC) algorithms.The focus is always on making the theory as well-motivated and accessible as possible, to allow students and readers to learn this fascinating subject as easily and painlessly as possible.

Features an introduction to probability theory using measure theory. This work provides proofs of the essential introductory results and presents the measure theory and mathematical details in terms of intuitive probabilistic concepts, rather than as separate, imposing subjects.

This book develops appreciation of the ingenuity involved in the mathematical treatment of random phenomena, and of the power of the mathematical methods employed in the solution of applied problems. It is intended to students interested in applications of probability to their disciplines.

This textbook is an introduction to rigorous probability theory using measure theory. It provides rigorous, complete proofs of all the essential introductory mathematical results of probability theory and measure theory. More advanced or specialized areas are entirely omitted or only hinted at. For example, the text includes a complete proof of the classical central limit theorem, including the necessary continuity theorem for characteristic functions, but the more general Lindeberg central limit theorem is only outlined and is not proved. Similarly, all necessary facts from measure theory are proved before they are used, but more abstract or advanced measure theory results are not included. Furthermore, measure theory is discussed as much as possible purely in terms of probability, as opposed to being treated as a separate subject which must be mastered before probability theory can be understood.

The ultimate objective of this book is to present a panoramic view of the main stochastic processes which have an impact on applications, with complete proofs and exercises. Random processes play a central role in the applied sciences, including operations research, insurance, finance, biology, physics, computer and communications networks, and signal processing. In order to help the reader to reach a level of technical autonomy sufficient to understand the presented models, this book includes a reasonable dose of probability theory. On the other hand, the study of stochastic processes gives an opportunity to apply the main theoretical results of probability theory beyond classroom examples and in a non-trivial manner that makes this discipline look more attractive to the applications-oriented student. One can distinguish three parts of this book. The first four chapters are about probability theory, Chapters 5 to 8 concern random sequences, or discrete-time stochastic processes, and the rest of the book focuses on stochastic processes and point processes. There is sufficient modularity for the instructor or the self-teaching reader to design a course or a study program adapted to her/his specific needs. This book is in a large measure self-contained.

A First Course in Stochastic Processes focuses on several principal areas of stochastic processes and the diversity of applications of stochastic processes, including Markov chains, Brownian motion, and Poisson processes. The publication first takes a look at the elements of stochastic processes, Markov chains, and the basic limit theorem of Markov chains and applications. Discussions focus on criteria for recurrence, absorption probabilities, discrete renewal equation, classification of states of a Markov chain, and review of basic terminologies and properties of random variables and distribution functions. The text then examines algebraic methods in Markov chains and ratio theorems of transition probabilities and applications. The manuscript elaborates on the sums of independent random variables as a Markov chain, classical examples of continuous time Markov chains, and continuous time Markov chains. Topics include differentiability properties of transition probabilities, birth and death processes with absorbing states, general pure birth processes and Poisson processes, and recurrence properties of sums of independent random variables. The book then ponders on Brownian motion, compounding stochastic processes, and deterministic and stochastic genetic and ecological processes. The publication is a valuable source of information for readers interested in stochastic processes.

Building upon the previous editions, this textbook is a first course in stochastic processes taken by undergraduate and graduate students (MS and PhD students from math, statistics, economics, computer science, engineering, and finance departments) who have had a course in probability theory. It covers Markov chains in discrete and continuous time, Poisson processes, renewal processes, martingales, and option pricing. One can only learn a subject by seeing it in action, so there are a large number of examples and more than 300 carefully chosen exercises to deepen the reader’s understanding. Drawing from teaching experience and student feedback, there are many new examples and problems with solutions that use TI-83 to eliminate the tedious details of solving linear equations by hand, and the collection of exercises is much improved, with many more biological examples. Originally included in previous editions, material too advanced for this first course in stochastic processes has been eliminated while treatment of other topics useful for applications has been expanded. In addition, the ordering of topics has been improved; for example, the difficult subject of martingales is delayed until its usefulness can be applied in the treatment of mathematical finance.