Researchers at the the University of Tokyo have developed an evasive maneuver assist (EMA) system using an array of Mikrotron high-speed cameras capable of communicating with each other via a wireless network. The system has a wide field-of-view (FOV) and high responsiveness, enabling vehicles to receive real-time information about potential dangers around them, thus allowing them to perform evasive maneuvering as quickly as possible to prevent accidents and reduce casualties.
One of the most challenging situations for drivers is avoiding collision with an obstacle that abruptly appears in front of the vehicle. This occurs because humans require a longer time to perform an evasive maneuver after sensing danger. The University of Toyko researchers were investigating methods of assisting drivers in performing evasive maneuvers at the exact moment when instantaneous reaction is required. One EMA technique, having cameras on board the vehicles, suffers from a limited field-of-view (FOV) and cannot observe pedestrians or motorcycles in a dead spot of the onboard camera. Another technique, vehicle-to-vehicle EMA communications, notifies drivers of each other’s position. However, this type of system requires virtually all vehicles be equipped with an alert and notification unit. In addition, the accuracy of GPS for vehicle navigation has room for improvement.
Multiple synchronized cameras
A single camera has limited object-tracking capability in large working areas, or whenever a target enters and leaves its FOV rapidly. In contrast, using multiple precisely synchronized high-speed cameras in the system provides a broader FOV that can successfully track a target even under extreme conditions, such as when the target moves rapidly across the FOV of each camera. Owing to the high acquisition rate, the system also recognizes events immediately after they occur around a tracking target, which is critical where even a slight latency can have adverse effects.
The researchers developed a 12V networked high-speed vision system composed of Mikrotron EoSens CMOS monochrome cameras (pictured) aligned to cover all areas of interest. The two CameraLink cameras were connected via a wi-fi network to enable them to communicate with one another, with each camera forming a node with its processing unit. All nodes were synchronized to sub-millisecond order using a software-defined precision time protocol. The cameras can robustly track objects moving around an entire space of interest and recognize events in every frame.
To test the system, researchers built a 1/10-scale robotic car platform featuring a real-time operating system to control steering and wi-fi communication modules. Two high-speed Mikrotron cameras were set at 600fps (frames per second) and connected to a workstation for image processing. One of the two workstations was set to transmit an obstacle map containing 50 obstacles.
Obstacles were placed by researchers at a fixed point. A line was drawn indicating the threshold distance of 800mm. Once the robotic vehicle reached the line, it performed evasive maneuvering. The cameras recognized the position of the vehicle during its entire path, including the area in which the FOVs overlapped, while the robotic car was traveling at 72km/h. Mikrotron high-speed cameras detected the moment the vehicle crossed the threshold line. The results clearly showed that it is critical to detect dangers as soon as possible to perform evasive maneuvering safely, and therefore extremely high frame rates are required. Similar testing on a camera set at 30fps failed to detect dangers fast enough to avoid them.
Researchers are now working on ways to reduce communication latency by introducing low-latency real-time wireless communication techniques and developing new applications beyond EMA based on the proposed system.