Over the past year, researchers from the University of California, Irvine (UCI) — including Professors Joshua Garcia and Qi Alfred Chen, along with graduate students Yuqi Huai, Yuntianyi Chen, Chi Zhang, and Xiang Liao — have made significant contributions to the advancement and evaluation of autonomous driving systems through a collaborative effort between the Software Aurora Lab (SORA) and the AS²Guard Research Group. With a particular focus on scenario generation and software testing, their work spans academic research, tool development, and active participation in open-source communities such as the Autoware Foundation. Their efforts reflect a broader goal: improving the safety, reliability, and transparency of autonomous driving systems through rigorous engineering practices and collaborative engagement.

One of the key events this year was a local workshop hosted at UCI in March 2025. The workshop brought together researchers from multiple institutions with the goal of eliciting requirements for a shared, cloud-based research infrastructure to support the development and testing of autonomous driving systems. Rather than focusing on a particular software stack, the workshop centered on identifying technical, logistical, and collaborative needs that such an infrastructure must address. Participants shared perspectives on scenario generation, simulation at scale, data management, and tool interoperability—laying the groundwork for a future platform that could support reproducible, cross-institutional research in autonomous driving systems.

Complementing this effort was UCI’s organization of the SE4ADS (Software Engineering for Autonomous Driving Systems) workshop at ICSE 2025. SE4ADS serves as a growing forum for advancing software engineering research tailored to the needs of autonomous driving systems. The 2025 edition featured work on simulation-based testing, requirements integration, and safety certification. Discussions also addressed broader concerns around responsible software practices and long-term maintainability, particularly in the context of open-source autonomous systems such as Autoware. The workshop underscored a shared commitment to developing engineering foundations that can support the unique complexity and risk profile of autonomous software.

With these community needs in mind, Yuqi is now leading the design and development of a Cloud-based Autonomous Driving Systems Research Infrastructure (CADRI). Building on insights from both the UCI-hosted workshop and the SE4ADS forum, this effort aims to create a scalable, interoperable platform that supports reproducible experimentation. A key advantage of the cloud-based approach is its ability to significantly reduce upfront costs, allowing researchers to perform large-scale testing and development without substantial investment in specialized hardware. This initiative builds on Yuqi’s earlier work in scenario-based testing, including DoppelTest [1] and scenoRITA [2], two frameworks for generating scenario-based tests. He also maintains a key dependency for the SVL simulator, helping ensure that the simulation tool remains accessible to the research community. In parallel, Xiang has been working on migrating tools originally developed for other ADS platforms onto Autoware, thereby broadening tool compatibility and reinforcing Autoware’s role as an open-source foundation for reproducible research.

In Yuntianyi’s latest research, he has been emphasizing Autoware as he led the development of ConfVE [4], a tool designed to identify failures in autonomous driving systems that arise from alternative configurations. Misconfiguration is a known risk factor in real-world deployments, and ConfVE aims to prevent such issues by identifying inconsistent or unsafe parameter combinations early in the development cycle. As part of this approach, Yuntianyi also developed a Scenario Record Analyzer—an automated tool capable of detecting nine distinct types of violations in Autoware driving scenario records, providing a robust mechanism for validating ADS behavior against safety and performance requirements. This work leveraged HD map and scenario data from Autoware Evaluator, obtained through a collaboration with the Autoware Operational Design Domain (ODD) Working Group. The partnership provided access to realistic, systematically generated test scenarios that reflect the ODD characteristics of Autoware’s target deployment environments, enabling ConfVE and the Scenario Record Analyzer to be validated under conditions closely resembling real-world usage. More recently, Yuntianyi presented A Comprehensive Study of Bug-Fix Patterns in Autonomous Driving Systems [5] at FSE 2025. This large-scale analysis examined over 1,300 real-world bug-fix instances from two leading open-source platforms (i.e., Autoware and Apollo) and introduced a taxonomy encompassing 15 syntactic and 27 semantic bug-fix patterns, capturing both code-level changes (e.g., conditional modifications, data structure corrections) and domain-specific modifications (e.g., path planning optimization, module integration and interaction). Yuntianyi’s work on ConfVE and the bug-fix pattern benchmark also contributes to the CADRI project, where he serves as a project leader. His contributions provide foundational components for the Toolkit Service, enrich the ADS analytics oracles, and supply a curated dataset repository, thereby strengthening CADRI’s capability to support comprehensive analysis, testing, and improvement of autonomous driving systems.

Besides the research, Yuntianyi and Yuqi contributed to the DevOps Dojo project within the Autoware OpenADKit Working Group. As part of this effort, they refactored approximately 15% of the total Autoware ROS nodes, enhancing maintainability and consistency in the codebase. Yuntianyi also developed an automated configuration refactoring tool for Autoware ROS nodes, enabling developers to standardize and update configurations more efficiently. This tool has accelerated the development workflow, reduced manual intervention, and improved configuration reliability across the Autoware ecosystem.

While Josh approaches autonomous driving from a software engineering perspective, focusing on faults that affect system reliability and correctness, Alfred brings a security lens to the field, concentrating on vulnerabilities in autonomous vehicles. Specifically, Alfred’s team focused on evaluating the robustness of autonomous driving systems by leveraging component-level vulnerabilities using methods such as adversarial scenarios, patches, and objects. Their efforts have contributed to a Platform for Auto-driving Safety and Security (PASS) [6], a modular and extensible simulation-based evaluation framework specifically for evaluating system-level effectiveness of existing attacks or defenses across different autonomous driving models. Building on top of PASS, Chi has been spending his recent efforts on designing and developing an adversarial scenario generation framework for Autoware using the CARLA simulation environment.

With the growing complexity and expanding deployment ambitions, the need for rigorous, collaborative, and scalable engineering practices in autonomous driving systems has never been more urgent. Josh and Alfred’s teams are helping to meet this need by integrating empirical insights with tool development, infrastructure planning, and community engagement. Their work, ranging from scenario-based test generation to large-scale bug fix analyses, demonstrates how software engineering research can directly contribute to the development of safer and more reliable autonomous systems. Through close collaboration with the Autoware Foundation and a commitment to open, reproducible experimentation via efforts like CADRI, they are contributing essential building blocks for a more robust, transparent, and evidence-driven research ecosystem in autonomous driving systems.

References

[1] Yuqi Huai, Yuntianyi Chen, Sumaya Almanee, Tuan Ngo, Xiang Liao, Ziwen Wan, Qi Alfred Chen, and Joshua Garcia. 2023. Doppelgänger Test Generation for Revealing Bugs in Autonomous Driving Software. In Proceedings of the 45th International Conference on Software Engineering (ICSE ’23). IEEE Press, 2591–2603. https://doi.org/10.1109/ICSE48619.2023.00216

[2] Yuqi Huai, Sumaya Almanee, Yuntianyi Chen, Xiafa Wu, Qi Alfred Chen, and Joshua Garcia, “scenoRITA: Generating Diverse, Fully Mutable, Test Scenarios for Autonomous Vehicle Planning,” in IEEE Transactions on Software Engineering, vol. 49, no. 10, pp. 4656-4676, 1 Oct. 2023, doi: 10.1109/TSE.2023.3309610.

[3] Yuqi Huai, 2023, SORA SVL Server. Available at https://github.com/YuqiHuai/SORA-SVL (Accessed: 30 June 2025).

[4] Yuntianyi Chen, Yuqi Huai, Shilong Li, Changnam Hong, and Joshua Garcia. 2024. Misconfiguration Software Testing for Failure Emergence in Autonomous Driving Systems. Proc. ACM Softw. Eng. 1, FSE, Article 85 (July 2024), 24 pages. https://doi.org/10.1145/3660792

[5] Yuntianyi Chen, Yuqi Huai, Yirui He, Shilong Li, Changnam Hong, Qi Alfred Chen, and Joshua Garcia. 2025. A Comprehensive Study of Bug-Fix Patterns in Autonomous Driving Systems. Proc. ACM Softw. Eng. 2, FSE, Article FSE018 (July 2025), 23 pages. https://doi.org/10.1145/3715733

[6] Hu, Zhisheng, Shen, Junjie, Guo, Shengjian, Zhang, Xinyang, Zhong, Zhenyu, Chen, Qi Alfred, and Li, Kang. PASS: A System-Driven Evaluation Platform for Autonomous Driving Safety and Security. Retrieved from https://par.nsf.gov/biblio/10359464. NDSS Workshop on Automotive and Autonomous Vehicle Security (AutoSec).