Stochastic Processes I

MATH 4221

Fall 2013

Skiles 254

Tu / Th, 9:35 - 10:55 pm


Will Perkins

Office: Skiles 017

Office Hours: Thursday, 11:00am - 1:00pm , or by appointment

Course Information

Main Topics of the Course:

  1. Basic Probability
  2. Expectation and Variance
  3. Law of Large Numbers and Convergence of Random Variables
  4. Conditional Probability
  5. Central Limit Theorem and Characteristic Functions
  6. Simple Random Walk and Branching Processes
  7. Martingales and Conditional Expectation
  8. Markov Chains


Required text: Grimmett and Stirzaker Probability and Random Processes

A useful accompanying book of exercises: One Thousand Exercises in Probability


Your grade will be determined by mastering the 8 course topics listed above. Grading is very simple: if you master all 8, you get an A; 6 or 7 a B; 4-5 a C; 2-3 a D; 0-1 an F.

Mastering a topic means understanding it from all angles. You should understand the definitions, theorems, and examples we've discussed in class. You should be able to apply the theorems and methods to problems you've never seen before (and not just be able to do problems of the same type you've seen). Understanding a theorem means understanding how it can be used (and knowing some specific examples); understanding why each of its conditions is necessary (and knowing counterexamples); understand why its conclusion cannot be made more strong.

There will be several ways to demonstrate that you've mastered a topic:

  1. An oral quiz.
  2. We will have two tests scheduled, and on each you will have the chance to show mastery of four topics.
  3. The final exam will be a last chance for any topics you still have left.

Once you've showed that you've learned a topic, you're done with it for the whole semester.

Here's link to Georgia Tech's honor code (please read it). It applies to the tests and final exam, but for everything else in the class (including the homework!) I encourage you to work together.


I will assign homework every two weeks. I encourage you to work on the problems alone at first, then check answers with each other, and explain the parts you don't understand to each other. You can use the piazza discussion site to do this. I will not collect or grade the homework, but I expect you to finish and understand all of the problems. You will need to understand all the solutions to pass the oral quizzes.

Oral Quizzes

Please read the rules carefully. I've designed them to encourage you to keep up with the course, while still allowing you to re-learn material you might have missed on a previous quiz or test.

  1. You may take 3 oral quizzes: one before September 26, one between Sep. 27 and October 31, and one after October 31 but before November 26.
  2. You may choose as many topics as you want to be tested on.
  3. You can retake topics.
  4. Each topic will be graded separately. You either pass (shown mastery) or not.
  5. The first question for each topic will be chosen from the homework questions. You must answer this 100% correctly. This is designed to make sure you do the homework.
  6. The other questions will be questions you have not seen before, but I will provide sample questions on the website.
  7. You should certainly know all the deifintions and theorems listed in the topic descriptions. "Knowing" means knowing the precise mathematical formualtions, knowing examples and counterexamples, and knowing how to apply theorems or methods to unfamiliar problems.

Tests & Final Exam

  1. Test #1: October 8, Topics 1-4
  2. Test #2: November 19, Topics 5-8
  3. Final Exam: December 12, 8:00 am - 10:50 am in the usual classroom, All topics
  4. Like the oral quizzes, the tests will be graded topic by topic, "mastered" or "not mastered".

Course Materials

In addition to the textbook, I will post lecture notes and occasional videos for the topics that aren't covered in the book. You will be responsible for knowing anything that we cover in class, whether or not it is in the textbook, but not responsible for the parts of the book that we don't cover in class.

Course Discussion Site

We will be using a website called as our course discussion site. Sign-up here. I'd like you all to be active on the site, asking questions, answering questions, and helping each other learn the material. Here are some possible uses:

  1. Any question about the class, an assignment, or the material you should ask on the discussion site instead of emailing me. That way everyone can see the answer.
  2. I encourage you to discuss the homework questions on the site.

  • Aug 20 Introduction and Probability Spaces

    Sections 1.2 - 1.3. Some additonal notes

  • Aug 22 Sets and Probabilities

    Sections 1.3 -1.5

  • Aug 27 Random Variables

    Sections 2.1, 2.3, 3.1

  • Aug 29 Expectation

    Sections 3.3, 3.4

  • Sep 3 Simple Random Walk & Asymptotics

    Sections 3.9, 3.10

  • Sep 5 no class

    Make-up 8:30 am, Tues Sep 10 Skiles 005

  • Sep 10 First and Second Moment Methods

  • Sep 12 Law of Large Numbers

    Sections 5.10, 7.4

  • Sep 17 Oral Quiz practice

  • Sep 19 no class

    Make-up 8:30 am, Tues Sep 24 Skiles 005

  • Sep 24 SLLN; Conditional Probability

  • Sep 26 Generating Functions

    Sections 5.1, 5.2, 5.7, 5.8

  • Oct 1 Characteristic Functions and the Central Limit Theorem

    Sections 5.10

  • Oct 3 Statistical Method

  • Oct 8 Test #1: Topics 1-4

  • Oct 10 Poisson Convergence and Random Graphs

  • Oct 15 Fall break : no class

  • Oct 17 Branching Processes

  • Oct 22 Markov Chains I

  • Oct 24 Markov Chains II

  • Oct 29 Martingales and Conditional Expectation

  • Oct 31 Martingale Convergence

  • Nov 5 Azuma's Inequality

  • Nov 7 Stationary Distributions

  • Nov 12MCMC

  • Nov 14 Eigenvalues and Markov Chains

  • Nov 19 Test #2: Topics 5-8

  • Nov 21 Poisson Processes

  • Nov 26 Large Deviations

  • Nov 28 No class - Thanksgiving Break

  • Dec 3Percolation

  • Dec 5 Review

  • Dec 12 Final Exam

    8:00 -10:50 am