Stochastic Processes I

Schedule

• Aug 20 Course basics; What is a stochastic Process?
• Aug 22 Probability Review: Outcomes, Events, Probability

  Chapter 1

• Homework 1, due August 30.
• Aug 24 Random Variables

  Chapter 2

• Aug 27 Expectation & Independence

  Chapter 3.1 - 3.3

• Aug 29 Indicator Random Variables

  Chapter 3.4 - 3.5

• Aug 31 Simple random walk

  Chapter 3.8 - 3.9

• Sep 5 Asymptotics, Conditional Distributions

  Chapter 3.6 - 3.7

• Homework 2, due September 12.
• Sep 7 The Ballot Theorem & More Random Walk

  Chapter 3.10

• Sep 10 Markov's Inequality, Chebyshev's Inequality

• Sep 12 Law of Large Numbers; Convergence of Random Variables

• Sep 14 Test #1 on Basic Probability, Expectation and Variance, & Simple Random Walk

• Test #1 Answers
• Sep 17 Continuous Random Variables Part I

• Sep 19 Graphs and Random Graphs

• Sep 21 Almost Sure Convergence, Strong Law of Large Numbers

• Homework 3, due October 1.
• Sep 24 Characteristic Functions

• Sep 26 Central Limit Theorem

• Sep 28 More on Limit Theorems

• Oct 1 Large Deviations

• Oct 3 Review

• Oct 5 Test #2: Limit Theorems, Convergence of Random Variables, Conditional Distributions

• Test #2 Answers
• Oct 8 Branching Processes

• Oct 10 Generating Functions

• Oct 12 more Generating Functions

• Oct 17 Transience and Recurrence

• Oct 19 Transience and Recurrence pt 2

• Oct 22 Markov Chains

• Oct 24 Classification of States

• Oct 26 Classification of Chains

• Oct 29 Stationary Distributions

• Oct 31 Review

• Nov 2 Test #3: Branching Processes, Generating & Characteristic Functions

• Test #3 Answers
• Nov 5 Limit Theorem for Markov Chains

• Nov 7 Reversibility

• Nov 9 Perron-Frobenius

• Nov 12 Mixing Times

• Nov 14 Markov Chain Monte Carlo

• Nov 16 Martingales

• Nov 19 Prediction and Conditional Expectation

• Nov 21 no class, enjoy your Thanksgiving

• Nov 26 Martingale Convergence

• Nov 28 Azuma's Inequality

• Nov 30 Test #4: Markov Chains, Martingales

• Test #4 Answers
• Dec 3 Probability Fundamentals

• Dec 5 Limit Theorems

• Dec 7 Stochastic Processes

Some goals of the course

1. Understand and enjoy the basics of random walks and random processes
2. Work like a mathematician: learn to ask good mathematical questions
3. Learn in a self-motivated and self-directed way
4. Learn to explain and present math clearly and precisely
5. Work hard and have fun

Course Information

Main Topics of the Course:

1. Basic Probability (sections 1.1 - 1.5, 2.1, 2.3, 2.5, 3.1, 3.2, 3.5, 4.1)
2. Expectation and Variance (3.3, 3.4, 4.3, 4.7, 5.6)
3. Simple Random Walk (3.9, 3.10, 5.3)
4. Conditional Distributions (3.2, 3.6, 3.7, 4.2, 4.5, 4.6)
5. Convergence of Random Variables (5.10, 7.1, 7.2, 7.3)
6. Limit Theorems (2.2, 3.8, 4.4, 4.8, 4.12, 5.10, 5.11, 7.4, 7.5)
7. Branching Processes (5.4, 6.7, 6.8)
8. Generating Functions and Characteristic Functions (5.1, 5.7, 5.8, 5.9)
9. Markov Chains (6.1 - 6.6, 6.14, 8.1 - 8.3)
10. Martingales (7.7 - 7.9, 12.1 - 12.5)

Textbook:

Required text: Grimmett and Stirzaker Probability and Random Processes

A useful accompanying book of exercises: One Thousand Exercises in Probability

Grading:

Your grade will be determined by mastering the 10 course topics listed above. Grading is very simple: if you master 9 or 10, you get an A; 7 or 8 a B; 4-6 a C; 2-3 a D; 0-1 an F.

Mastering a topic means understanding it from all angles. You should understand the definitions, theorems, and examples we've discussed in class. You should be able to apply the theorems and methods to problems you've never seen before (and not just be able to do problems of the same type you've seen). Understanding a theorem means understanding how it can be used (know some examples); understanding why each of its conditions is necessary (and know counterexamples); understand why its conclusion cannot be made more strong.

There will be several ways to demonstrate that you've mastered a topic:

1. A short oral quiz (10 minutes) at the time of your choosing. You will have 2 attempts for each topic.
2. We will have 4 tests scheduled, and on each you will have the chance to show mastery of 2 - 4 topics.
3. The final exam will be a last chance for any topics you still have left.

Once you've showed that you've learned a topic, you're done with it for the whole semester. The tests are an indication of where you are and a guide so you don't fall behind. But you are welcome (and encouraged) to pass all of the topics with the oral quizzes whenever you feel you have learned the topics.

Here's link to Georgia Tech's honor code (please read it). It applies to the tests and final exam, but for everything else in the class I encourage you to work together.

Course Discussion Site

We will be using a website called piazza.com as our course discussion site. Sign-up here. I'd like you all to be active on the site, asking questions, answering questions, and helping each other learn the material. Here are some possible uses:

1. Any question about the class, an assignment, or the material you should ask on the discussion site instead of emailing me. That way everyone can see the answer.
2. I encourage you to post your answers to the homework questions on the site. If someone else has posted the same answer as yours you can ``agree'', or if you have gotten a different answer you can explain yours. I will `endorse' the best answer I see.