The CAPS AI Policy and Strategy skills course is an immersive six-part seminar that trains participants in the role of a fictional Forward-Deployed Policy Engineer tasked with overseeing AI systems deployed in an ongoing conflict zone.
The course begins with an AI bootcamp and quickly moves participants “into the field” to confront case studies rooted in real needs in the test environment of Afghanistan. Taught at Johns Hopkins SAIS in the Spring 2019 semester.
We're excited that CAPS will continue to be taught at Johns Hopkins SAIS in the Fall 2019 semester. We're currently updating our curriculum and lecture materials. Therefore, some of this repo will be "under construction".
CAPS takes place over three weeks, with two 90 minute sessions per week. The first session will be on Wednesday, April 3.
-
Wednesdays 6:00–7:30 pm in Nitze 507
-
Fridays 6:00–7:30 pm in BOB 736
📧 For enrollment or questions, contact capsseminar@gmail.com
A pdf version of the syllabus can be found here (last updated, April 01 2019).
All materials not linked below are forthcoming. All can also be found in and downloaded from the Docs folder.
📘 Lecture 1 - Introduction to Human Factors and Reinforcement Learning
-
Guiding question: How do human teams make decisions and how does this decision-making compare to autonomous decision processes, such as those of reinforcement learning algorithms?
-
Topics covered: Learning from interaction; Decision-making in human teams; Reinforcement learning; Markov decision processes; Bellman equation
-
Case: Learning in a counterinsurgency team
📘 Lecture 2 - Rule-Based Decision Making in a Fuzzy World
-
Guiding question: How can we transform fuzzy descriptions of specialized human performances into computable knowledge for an autonomous system to act on?
-
Topics covered: Rule-based systems; Finite-state machines; Basic search algorithms; State space complexity; System requirements in fuzzy environments
-
Case: Designing and evaluating a rule-based system to clear a conflict zone
📘 Lecture 3 - Learning to Make Decisions with and without a Model of the World
-
Guiding question: How can we conceptualize core differences in learning architectures and apply this knowledge to augment partial observations of a reinforcement learner’s performance?
-
Topics covered: Q-learning; Value iteration; Basic inverse reinforcement learning; Black box problems
-
Case: Determining and evaluating possible learning architectures of an enemy drone
-
Notes | Slides | Case: Base Code and Supplementary Code
📘 Lecture 4 - Guiding Reinforcement Learners through Human Control
-
Guiding question: How can we use human input throughout the reinforcement learning process and during deployment to ensure optimal system performance and what are the trade-offs of this approach?
-
Topics covered: Human-in the-loop reinforcement learning; Shared autonomy; Reward shaping
-
Case: Improving the operation of a semi-autonomous supply convoy in a contested environment
-
Notes | Slides | Case
📘 Lecture 5 - Making Sound Long-Term Predictions about AI
-
Guiding question: How can we synthesize partial observations into stable predictions about the longterm development of AI and its impact on operational ecosystems?
-
Topics covered: Methods of analysis and prediction
-
Case: Developing recommendations on implementing autonomous decision-making into a counterinsurgency campaign
📘 Lecture 6 - Participant Presentation and Debriefing with Senior Policymaker
Guiding question: How can we provide meaningful insights into a complex, technical domain for a senior policymaker to determine high-level strategy
- Details for guest senior policymaker TBA.
CAPS is pre-approved as an official SAIS skills course, meaning that if you attend all of the sessions, you obtain certification for the course on your transcript.
For the course to appear on one’s trascript, they must attend all sessions.
CAPS is created and taught by SAIS MAs Leo Klenner, Henry Fung, Cory Combs and JJ Lee.
The course is sponsored by Sarah Sewall, Former Undersecretary of State for Civilian Secuirty, Democracy and Human Rights (2014-2017) and Speyer Family Foundation Distinguished Scholar at the Henry A. Kissinger Center for Global Affairs.
CAPS is funded through a grant from Johns Hopkins Technology Ventures.
📧 To contact us, reach out to capsseminar@gmail.com