Christopher D. Hsu

I am a PhD student in the Electrical and Systems Engineering department and the GRASP Robotics Laboratory at the University of Pennsylvania, where I work on multi-robot systems. My PhD advisor is Pratik Chaudhari, Assistant Professor at Penn in ESE with a secondary appointment in CIS.

Currently, I am also a Civilian Research Engineer at DEVCOM Army Research Laboratory where our mission is to operationalize science for tranformational overmatch.

As a DoD SMART Scholar, I received a MSE in Robotics from the University of Pennsylvania, where I was advised by George J. Pappas and Pratik Chaudhari. I have a BS in Mechanical Engineering from Villanova University, where I worked with C. Nataraj in the Villanova Center for Analytics of Dynamic Systems (VCADS).

GitHub  /  Google Scholar  /  LinkedIn

chsu8 at seas dot upenn dot edu

Research

I am interested in multi-robot systems, reinforcement learning, planning, controls, and bio-inspired systems.

A Model for Perimeter Defense Problems with Heterogeneous Teams

We introduce a model for multi-agent interaction problems to understand how a heterogeneous team of agents should organize its resources to tackle a heterogeneous team of attackers. This model is inspired by how the human immune system tackles a diverse set of pathogens. The key property of this model is “cross-reactivity” which enables a particular defender type to respond strongly to some attackers but weakly to a few different types of attackers. Due to this, the optimal defender distribution that minimizes the harm incurred by attackers is supported on a discrete set. This allows the defender team to allocate resources to a few types and yet tackle a large number of attacker types. We study this model in different settings to characterize a set of guiding principles for control problems with heterogeneous teams of agents, e.g., sensitivity of the harm to sub-optimal defender distributions, teams consisting of a small number of attackers and defenders, estimating and tackling an evolving attacker distribution, and competition between defenders that gives near-optimal behavior using decentralized computation of the control. We also compare this model with reinforcement-learned policies for the defender team.

Scalable Reinforcement Learning Policies for Multi-Agent Control

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments, ablation studies, and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

Design and source code from Leonid Keselman's website