This section is divided into recommended textbooks, readings by session, and course notes. There are no required textbooks for this course.

Recommended Textbooks

A list of relevant textbooks is given below:

Buy at Amazon Strogatz, Steven H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Boulder, CO: Westview Press, 2001. ISBN: 9780738204536.

Buy at Amazon Slotine, Jean-Jacques E., and Weiping Li. Applied Nonlinear Control. Upper Saddle River, NJ: Prentice Hall, 1991. ISBN: 9780130408907.

Buy at Amazon Fantoni, Isabelle, and Rogelio Lozano. Non-linear Control for Underactuated Mechanical Systems. New York, NY: Springer-Verlag, 2002. ISBN: 9781852334239.

Buy at Amazon Bertsekas, Dimitri P. Dynamic Programming and Optimal Control. 3rd ed. Vols. I and II. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083 (set).

Buy at Amazon Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. ISBN: 9780262193986.

Buy at Amazon Bertsekas, Dimitri P., and John N. Tsitsiklis. Neuro-Dynamic Programming. Nashua, NH: Athena Scientific, 1996. ISBN: 9781886529106.

Buy at Amazon LaValle, Steven M. Planning Algorithms. New York, NY: Cambridge University Press, 2006. ISBN: 9780521862059.

Readings by Session

The readings below come from the course notes "Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines."


Fully- vs. under-actuated systems


Chapter 1 and Appendix A
2 Nonlinear dynamics of the simple pendulum Chapter 2

Introduction to optimal control

Double-integrator examples

Chapter 9

Double integrator (cont.)

Quadratic regulator (Hamilton-Jacobi-Bellman (HJB) sufficiency), min-time control (Pontryagin)

Chapter 10
5 Dynamic programming and value interation: grid world, double integrator, and pendulum examples

Chapter 9 (cont.)

6 Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping Chapter 3
7 Acrobot and cart-pole (cont.) Chapter 3 (cont.)
8 Policy search: open-loop optimal control, direct methods, and indirect methods Chapter 12
9 Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) Chapter 12 (cont.)
10 Simple walking models: rimless wheel, compass gait, kneed compass gait Chapter 5
11 Feedback control for simple walking models Chapter 5 (cont.)
12 Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers Chapter 6
13 Motion planning: Dijkstra's, A-star Chapter 13
14 Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps Chapter 13 (cont.)
15 Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees Chapter 14
16 Function approximation and system identification Chapter 8 and Appendix B
17 Model systems with uncertainty: state distribution dynamics and state estimation Chapter 8 (cont.)
18 Stochastic optimal control Chapter 15
19 Aircraft Chapter 7
20 Swimming and flapping flight Chapter 7 (cont.)
21 Randomized policy gradient Chapter 17
22 Randomized policy gradient (cont.) Chapter 17 (cont.)
23 Model-free value methods: temporal difference learning and Q-learning Chapter 16

Actor-critic methods

Final project presentations

Chapter 18
25 Final project presentations  


Course Notes

Selected chapters from the course notes are available below. Updated revisions of the course notes are available here.

Front Title page, table of contents, and preface (PDF)
1 Fully actuated vs. underactuated systems (PDF)
I. Nonlinear dynamics and control
2 The simple pendulum (PDF)
3 The acrobot and cart-pole (PDF)
4 Manipulation
5 Walking (PDF)
6 Running
7 Flight
8 Model systems with stochasticity
II. Optimal control and motion planning
9 Dynamic programming (PDF)
10 Analytical optimal control with the Hamilton-Jacobi-Bellman sufficiency theorem (PDF)
11 Analytical optimal control with Pontryagin's minimum principle
12 Trajectory optimization (PDF)
13 Feasible motion planning
14 Global policies from local policies
15 Stochastic optimal control
16 Model-free value methods
17 Model-free policy search (PDF)
18 Actor-critic methods
IV. Applications and extensions
19 Learning case studies and course wrap-up

A. Robotics preliminaries (PDF)

B. Machine learning preliminaries

Back References (PDF)