The majority of the motoring world may be focusing on getting self-driving cars onto the road, but an effort to take autonomy onto alternative terrain is also well underway.
Drive settings, gear selection and maneuvering could all one day be handled by a computer. The aim, however, isn’t to take the fun out of off-roading, but to help develop smarter and safer AVs, whatever the weather.
This is the goal for Land Rover’s Cortex project, which is a bid to take learnings gleaned from the development of on-road autonomy, and apply it to alternative terrain. Using a 5D visualization system, the Cortex initiative, it is hoped, will deliver a car that can drive itself in any climatic condition, dealing with dirt, rain, ice, snow and fog. It also aims to surmount obstacles of infinite shape and size, challenging gradient changes, rock crawls or water hazards, and the lack of road markings and signs – hindrances that can give even the most seasoned off-roader cold sweats.
“We are working on a range of connected and autonomous vehicle technologies that will ensure our vehicles remain the most capable on- and off-road,” says Nigel Clarke, senior manager of Land Rover’s Level 5 All Surface Advanced Sensor department, and leader of the Cortex project.
Lessons learned from this project will also help Land Rover produce vehicles that can handle on-road driving in potentially perilous situations such as when road signs and markings are faded or missing, and when sensors are covered by dirt.
“We are developing a world-first 5D data technique, which combines acoustic, video, radar, light detection and distance-sensing data in real time,” explains Clarke. “This means the car has more awareness of the environment it is in, compared with on-road trials. Machine-learning techniques are being used to help the car interpret this vast amount of data, so that it can behave in an increasingly sophisticated way and handle any weather condition or terrain.”
Military maneuvers
Although Land Rover has only just begun to show its autonomous off-roading hand, others have been working on the technology for more than a decade. DARPA, the Defense Advanced Research Projects Agency of the US Department of Defense, kick-started the development of off-road autonomy with the Grand Challenge, back in 2004.
With a US$1m prize on offer, teams were invited to design and build a vehicle that could autonomously navigate a 142-mile (230km) course through the desert from Barstow, California, to Primm, Nevada. This may sound ambitious even by today’s autonomous standards – and indeed it was. The most successful of the competition’s 15 finalists managed just 7.5 miles (12km).
However, it wasn’t long before passenger car manufacturers took interest. Volkswagen entered the DARPA Grand Challenge in 2005 with Stanford University, fielding a modified Touareg SUV called Stanley against 22 other finalists. Volkswagen and Stanford won the competition, completing the slightly shortened 132-mile (212km) course in six hours and 53 minutes.
As with today’s autonomous cars, the Touareg featured cameras, GPS and roof-mounted lidar, feeding data to computers that constructed a real-time 3D map of the car’s surroundings. Algorithms then worked out which route it should take, selecting good options from bad, with a mere 0.00002% chance of falling for a false positive.
For its development of project Cortex, Land Rover is working on technologies not dissimilar to those seen in the Nevada desert more than 14 years ago, and has chosen to collaborate with the University of Birmingham in the UK – a leader in research into radar and sensing technology for AVs – and Myrtle Software, machine-learning experts based in Cambridge, UK.
The University of Birmingham’s focus is on radar and its ability to function, regardless of rain, fog and poor light, all of which affect the cameras, infrared and lidar. “Our role is to examine different cognitive approaches to radar parameter setting and signal processing,” says Prof. Christopher Baker, chairman of the University of Birmingham’s Intelligent Sensor Systems department.
“Radar parameters can be changed on the fly to enhance the information gathered and improve the perception of the road ahead. This will require more advanced radar than is used today and intelligent interpretation of echoes to understand the imaged scene.”
Baker doesn’t shy away from the challenges faced by the development of off-road autonomy: “This is all weathers, all road types, on and off. Cameras won’t work in poor light or visibility, and infrared cameras and lidar won’t work in rain and fog, so radar has to do the job. Hence our emphasis on this sensor.”
Researchers at the Georgia Institute of Technology in the USA have also been making developments in this area with its study, Agile Autonomous Driving Using End-to-End Deep Imitation Learning, which trained a deep neural network in order to map high-dimensional observations to the continuous steering and throttle commands of its test vehicle.
“Off-road autonomy poses considerable challenges and characteristics,” explains Evangelos Theodorou, a professor at Georgia Tech and a member of the team on the project. “Vehicles have to move on very uneven terrains with variable morphology and traction conditions that are uncertain and can rapidly change. Also, the ability to predict the future in terms of morphology and terrain conditions is critical.”
Over the past four years the team has been working on a new unified approach to off-road autonomy navigation that solves the problem of when the road ahead is unmapped and when control is at its trickiest.
“Existing methods for autonomous navigation are based on a planning module that generates high-level trajectories using a simplified dynamics model, and a control module that tracks these trajectories. However, these architectures are inadequate for handling the dynamics of vehicles moving off-road and for generating agile behaviors,” says Theodorou.
“A key ingredient of our approach is its ability to take into consideration the full dynamics of the vehicle and its interactions with the environment,” continues Theodorou. “Our algorithms are capable of predicting the future and making adjustments based on these predictions in real time using state-of-the-art methods of stochastic control for decision making, deep learning for perception, and parallel computation for real-time performance.”
Recovery position
Redundancy is an important part of any autonomous system, helping the car navigate correctly when sensors are obscured or damaged. And for off-road autonomy this is especially important, given the increased likelihood of mud, dirt, snow and water covering the vehicle’s ‘eyes’ and ‘ears’.
“The solution to this problem consists of many possibilities,” says Theodorou. “Hardware solutions such as intelligent cleaning systems will play an important role, while algorithms that are able to reconstruct information using corrupted sensory data will be essential. Approaches such as generative adversarial networks can also be of great use. The promise of these approaches is that they recover information sensor failures,” explains Theodorou.
Project Cortex uses cross-learning techniques for situations where one sensor fails – such as when it is covered by mud. Clarke explains how the vehicle’s other sensors then use historically learned knowledge, or divert their attention away from non-critical systems, to allow the car to continue to make progress.
“A lot of work goes into determining the optimal location for sensors to avoid dirt,” he says. “Our water and dirt management team model dirt build-up and thickness to make recommendations to the aerodynamics team to minimize the impact on sensors.”
Although not yet finalized, the Cortex team is currently working to at least three levels of redundancy for any degree of coverage on the car. For example, any three of the vehicle’s six systems – camera, lidar, short-range radar, long-range radar, ultrasonic, GPS – could be taken off-line and it would still be able to drive itself.
Sensors are positioned to avoid damage from stones or gravel, or blockage by mud or water. Radar is mounted to the vehicle roof to capture 3D information on its surroundings, but if that is somehow obscured, the car can refer to a 2D map of its route ahead.
Applications for off-road autonomy
“There is a plethora of commercial applications for autonomous off-road vehicles,” says Theodorou. “These applications range from agriculture, tourism in remote areas, search and rescue missions, defense and, of course, mining.”
Applying autonomous off-roading to today’s human-driven vehicles could, Land Rover claims, make drivers feel safer while off-roading and give them greater confidence to navigate trickier situations. Indeed, Clarke suggests that an SUV with autonomous off-roading capabilities could take control of the driving if a driver was nervous in a certain situation: “It could assess the scenario and conduct the driving activity with confidence,” he says.
Land Rover has worked with the Red Cross since 1954, most recently developing a modified Discovery with an aerial reconnaissance drone launching from its roof to provide video of natural disasters. The car maker now has ambitions to use project Cortex for similar work. “Our humanitarian work is of the utmost importance,” says Clarke. “If we can deploy autonomous vehicles in a natural disaster, it might enable more people to be reached more quickly. One of the scenarios we’re placing a lot
of focus on is how the sensing system could cope with floods.”
Although the technology is still in its early days, the prospect of using artificial intelligence – like that employed by Cortex to navigate safely – is something the Red Cross is keen to explore further. “What’s interesting is the possible convergence between humanitarian principles and AI,” says Aarathi Krishnan, global futures and foresight coordinator, International Federation of Red Cross and Red Crescent Societies. “We are really keen to be involved in discussions around how AI is programmed to make decisions, especially those that will impact human lives.”
As with all forms of autonomous mobility, safety and a decent justification for developing and commercializing the technology will be key to Cortex’s long-term success. But even if the finished product remains unclear for now, Land Rover’s desire to take the technology off-road shows how applications for driverless systems extend far beyond the taxis of Silicon Valley.
By Alistair Charlton