In the second half of this tutorial, we introduce a number of open problems in statistical reinforcement learning. These include testing a parsimonious model against a nonparametric alternative; spatio-temporal decision problems; and doubly-robust methods for combining model-based and model-free methods. We introduce each problem and, in most cases, present partial solutions while identifying remaining technical and conceptual challenges.