# Markov Chains

### Fall 2012

#### M / W / F, 12:05 - 12:55 pm

Instructor

Will Perkins

wperkins3@math.gatech.edu

Office: Skiles 017

Office Hours: Wednesday 10 am - 12pm, or by appointment

Topics
• Sections: 6.1 - 6.6, 6.14
• Definitions:
1. Markov Chain
2. Homogeneous Markov Chain
3. Transition Matrix
4. n-step Transition Matrix
5. Recurrent State
6. Transient State
7. Null and positive recurrent
8. Periodic
9. Intercommunicating States
10. Closed
11. Irreducible
12. Stationary Distribution
13. Recurrence Time
14. Reversible
15. Total Variation Distance (3 definitions)
16. Mixing Time
17. Coupling of Two Markov Chains
18. Random Walk on a Graph
• Theorems: 6.1.5, 6.1.8, 6.2.3, 6.2.4, 6.2.5, 6.2.9, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.4.3, 6.4.6, 6.4.17, 6.5.4, 6.6.1, 6.14.9, Coupling Collison Time Bound on Mixing Time,
• Sample Questions:
1. What is the mixing time for a random walk on the complete graph with self loops?
2. Give upper and lower bounds for the mixing time of a random walk on two complete graphs of size n connected by a single edge.
3. Can a single Markov Chain have more than one stationary distribution? Why or why not?
4. Consider a two state MC with states A, B. p(A,A) =1/2, p(A, B) =1/2. P(B,B) =1/4, P(B,A) =3/4. What is the stationary distribution? What are the eigenvalues of the adjacency matrix?
5. Is a branching process a reversible Markov Chain? How about SRW?
6. Decompose the state space of the branching process MC.
7. Prove that 3d random walk is transient.
8. Prove that 2d random walk is null recurrent.
9. What is the stationary distribution of the random walk on the complete bipartite graph where the left partition has n vertices, the right 2n, and at each step there is probability 1/2 of remaining in the current state.
10. Give a graph whose random walk has period 3.
11. Give a Markov chain whose state space is not irreducible.
12. Give a Markov chain whose state space can be partitioned into two closed sets of states.
13. Consider m balls lying in n bins. At each step we pick a ball at random out of a bin, and throw it into a bin chosen uniformly at random. Is this a Markov Chain? What is the state space? Clasify the chain. What is the stationary distribution? Give upper and lower bounds for its mixing time.