speaker info
Parasara Sridhar Duggirala is an Assistant Professor in the Department of Computer Science at University of North Carolina at Chapel Hill. Prior to this, he was an Assistant Professor in the Computer Science and Engineering Department and UTC Institute for Advanced Systems Engineering at University of Connecticut from August 2015 – December 2018. He has received his PhD from University of Illinois at Urbana Champaign in 2015 and B.Tech from IIT Guwahati in 2009. His research interests span fields of Cyber-Physical Systems, Formal Methods, Control Theory, Hybrid Systems, Autonomy, Embedded and Real-Time Systems, and Probabilistic Systems. He has received Best Paper Award at International Conference on Embedded Software (EMSOFT) 2013, Most Promising Benchmark Result by Robert Bosh at ARCH Workshop in CPS Week 2015, and Best Paper Award at ARCH Workshop in CPS Week 2017. He was selected as a Young Researcher to attend the Heidelberg Laureate Forum in 2014.
ABstract
Autonomous vehicles (AVs) are already a reality and several prototypes and beta-testing deployments have driven several millions of miles. Despite such deployments, AVs have not even seen a fraction of the total number of miles driven annually or the various traffic configurations. However, evidence that an AV can successfully navigate a wide range of traffic scenarios is crucial not just for certification but also for building public trust. In this talk, we will investigate the various challenges involved in generating test scenarios for autonomous vehicles and build user trust. One of the challenges in designing autonomous vehicles (AV’s) is driving around humans (i.e. drivers, cyclists, pedestrians, etc.) In particular, the AV’s and the humans must have a common set of traffic rules to follow. In this talk, we first present a new approach to formalize and implement traffic rules. We use California’s DMV driver handbook as a working example. Our approach provides a straightforward mapping from the rules in the handbook to its formal model, and from the model to its implementation. To demonstrate the efficiency of this approach, we formally model the traffic rules in the logic programming paradigm of Answer Set Programming (ASP) using a programming language called Clingo. We then integrate these rules into CARLA, a virtual test bed environment for autonomous vehicles. We simulate the behavior of autonomous vehicles at four way and three way uncontrolled intersections by correct reasoning of right-of-way rules for autonomous vehicles in real time. As a result, the behaviors of autonomous vehicles under our controller are more realistic compared to CARLA’s default FIFO controller. This also improves the throughput of the traffic through the intersection.
Given a model of traffic rules, we then investigate techniques for automatically generating test scenarios. For the purposes of certification, we need test scenarios that are “wide” and “deep”. That is, an AV is not just expected to behave correctly in a wide range of scenarios, but also should react to nuances in variations of a scenario. We use two different techniques for addressing these two different challenges. For generating a wide range of behaviors, we propose a new coverage metric and a new fuzzing-based framework. Our main hypothesis is that code coverage is not an appropriate criteria for generating a diverse set of traffic-scenario test-cases. This renders off-the-shelf fuzzers (e.g. libFuzzer, AFL, etc) suboptimal, as they support code-coverage only. We propose a new coverage criteria based on high-level predicates derived from the end-to-end requirements of AVs, e.g. traffic rules. Using this new coverage criterion, we implement new fuzzing-based scenario generation techniques to automatically synthesize new test cases for AVs. For generating deep test case scenarios, we define a new notion of complexity of a given scenario. Given a test-case, we model the problem of generating a more complex test-case as a constraint-satisfaction search problem. The search variables are the changes to the given test-case, and the search constraints define a solution to the search problem. We have implemented our algorithms using the Scenic libraries and the CARLA simulator and generate test-cases for several 3-way and 4-way intersections with different topologies. Our experiments demonstrate that various AV implementations such as, CARLA’s autopilot, autopilot-plus-RSS (Responsibility-Sensitive Safety), and other end-to-end RL frameworks can fail to satisfy the traffic rules with increased number and complexity of test scenarios.