The ISS Scenario

ISS-Scenario is a comprehensive scenario library designed for testing Autonomous Driving Systems (ADSs) within the CARLA simulator. Built using the CARLA’s ScenarioRunner tool, the library provides a wide range of traffic scenarios with parameterized configurations to test ADSs in diverse driving conditions.

The library is compatible with CARLA versions up to 0.9.10.1. While other CARLA versions are not natively supported, they can be made compatible with minor modifications to the ScenarioRunner settings.

ISS-Scenario aims to empower developers and researchers with a rich set of scenarios that can simulate both routine and complex interactions, ensuring robust performance evaluation of ADSs.

Usage

To use ISS-Scenario, follow these steps:

  1. Install the CARLA simulator: download and set up the CARLA simulator (version 0.9.10.1 is recommended for optimal compatibility).
  2. Set up ScenarioRunner: ensure that the CARLA’s ScenarioRunner tool is properly configured. This tool is essential for generating and running the parameterized scenarios provided by ISS-Scenario.
  3. Select a scenario: browse the scenario library and select a scenario that fits your testing requirements. Scenarios are categorized based on key testing goals, such as obstacle recognition, intersection encounters, and emergency evasion.
  4. Configure parameters: modify the scenario parameters using configuration files. Options include:
    • environmental conditions (e.g., weather, time of day);
    • actor behaviors (e.g., vehicle speeds, pedestrian paths); and
    • traffic flow characteristics.
  5. Generate the scenario: use ScenarioRunner to load the selected scenario with your configuration into the CARLA simulator. This step initializes the simulation environment with all necessary parameters.
  6. Run the simulation: execute the scenario within CARLA and monitor the ADS’s performance.
  7. Evaluate the results: analyze the behavior of the ADS to identify strengths, weaknesses, and areas for improvement.

Demo: Intersection Encounter with a Red-Light Runner

One of the notable scenarios in ISS-Scenario is an intersection encounter where the ADS must respond to a vehicle running a red light. This critical scenario tests the ADS’s perception, decision-making, and reaction time in a high-risk situation.

In the demo below:

  • the scenario features a vehicle entering the intersection from the side while running a red light;
  • the ADS under test is Interfuser, which must detect the red-light runner and take appropriate evasive action to avoid a collision.

Scenario Layout

Layout of the intersection encounter scenario with a red-light runner

ADS Performance

The Interfuser ADS encountering a collision with the red-light runner

This demo highlights the challenges of high-stakes traffic scenarios and underscores the importance of rigorous testing to ensure ADS safety and reliability.

Supported Scenarios

The ISS-Scenario library includes the following categories of scenarios, with detailed parameterization for each type:

1. Obstacle Recognition

Scenarios designed to test the ADS’s ability to detect and respond to static obstacles. Examples include:

  • Straight-road obstacles: stationary objects blocking the ADS’s path on straight roads.
  • Intersection obstacles: obstacles appearing after a turn or at intersections.
  • Parked vehicles: simulating scenarios where improperly parked vehicles obstruct the road.

2. Intersection Encounters

Scenarios focusing on the ADS’s handling of complex interactions at intersections. Examples include:

  • Crossroad interactions: navigating traffic at four-way intersections.
  • Side street merges: vehicles entering the main road from side streets.
  • Unprotected turns: testing left and right turns in high-traffic conditions.

3. Pedestrian and Non-Motorized Vehicle Interactions

Scenarios involving interactions with vulnerable road users such as pedestrians and bicycles. Examples include:

  • Pedestrian crossings: testing ADS reactions to pedestrians crossing at crosswalks or jaywalking.
  • Bicycle interactions: cyclists traveling alongside the ADS or crossing intersections.
  • Mixed scenarios: combined interactions involving both pedestrians and bicycles in dynamic settings.

4. Surrounding Vehicle Interactions

These scenarios simulate traffic flow and dynamic behavior around the ADS. Examples include:

  • Lane merging: vehicles entering the ADS’s lane from adjacent lanes.
  • Sudden stops: vehicles ahead decelerating abruptly before continuing.
  • Highway scenarios: vehicles merging onto highways from on-ramps at varying speeds.

5. Emergency Evasion

Critical scenarios that test the ADS’s ability to respond rapidly and accurately. Examples include:

  • Sudden pedestrian appearances: pedestrians stepping onto the road without warning.
  • Vehicle skidding: nearby vehicles losing control and entering the ADS’s path.
  • Emergency braking: simulating scenarios requiring immediate stopping to avoid collisions.

Scenario library scale:
The ISS-Scenario library currently features over 100 scenarios, including various parameterized variants. This extensive collection ensures comprehensive testing coverage for a wide range of real-world driving challenges. Details of the actual content of the library can be found here.

Table of contents