Nobody likes to sit at a red light. But signalized intersections are not just a minor annoyance for drivers; vehicles consume fuel and emit greenhouse gases while waiting for the light to change.
What if motorists could time their journeys to arrive at the intersection when the light is green? While that might just be luck for a human driver, it could be more consistently achieved by an autonomous vehicle that uses artificial intelligence to control its speed.
In a new study, MIT researchers demonstrate a machine learning approach that can learn to control a fleet of autonomous vehicles as they approach and cross a signalized intersection in a way that keeps traffic flowing.
Using simulations, they found that their approach reduced fuel consumption and emissions while improving average vehicle speeds. The technique performs best if all the cars on the road are self-driving, but even if only 25% use their control algorithm, it still leads to substantial fuel and emissions benefits.
“It’s a really interesting place to intervene. Nobody’s life is better because they were stuck at an intersection. With many other climate change interventions, a difference in quality of life is expected, so there is a barrier to entry. The barrier here is much lower,” says lead author Cathy Wu, Gilbert W. Winslow Assistant Professor of Career Development in the Department of Civil and Environmental Engineering and Fellow of the Institute for Data, Systems, and Society. (IDSS) and the Laboratory of Information and Decision Systems (LIDS).
The lead author of the study is Vindula Jayawardana, a graduate student in LIDS and the Department of Electrical Engineering and Computer Science. The research will be presented at the European Control Conference.
Complexities of intersections
While a human can pass a green light without thinking too much about it, intersections can present billions of different scenarios depending on the number of lanes, the operation of signals, the number of vehicles and their speed, the presence of pedestrians and cyclists, etc.
Typical approaches to solving intersection control problems use mathematical models to solve a simple and ideal intersection. It looks good on paper, but probably won’t hold up in the real world, where traffic patterns are often about as messy as they come.
Wu and Jayawardana shifted gears and tackled the problem using a model-free technique known as deep reinforcement learning. Reinforcement learning is a method of trial and error in which the control algorithm learns to make a sequence of decisions. He is rewarded when he finds a good sequence. With deep reinforcement learning, the algorithm exploits hypotheses learned by a neural network to find shortcuts to good sequences, even if there are billions of possibilities.
This is useful for solving a long horizon problem like this; the control algorithm must issue more than 500 acceleration instructions to a vehicle over an extended period of time, Wu says.
“And we need to get the sequence right before we know we’ve done a good job of mitigating the emissions and getting to the intersection at a good speed,” she adds.
But there is an additional wrinkle. The researchers want the system to learn a strategy that reduces fuel consumption and limits the impact on travel time. These objectives can be contradictory.
“To reduce travel time, we want the car to go fast, but to reduce emissions, we want the car to slow down or not move at all. These competing rewards can be very confusing for the learning agent” , says Wu.
Although it’s difficult to solve this problem in its full generality, the researchers found a workaround using a technique known as reward shaping. Together with reward shaping, they give the system domain knowledge that it is unable to learn on its own. In this case, they penalized the system each time the vehicle came to a complete stop, so that it learned to avoid this action.
Once they developed an effective control algorithm, they evaluated it using a traffic simulation platform with a single intersection. The control algorithm is applied to a fleet of connected autonomous vehicles, which can communicate with upcoming traffic lights to receive signal phase and timing information and observe their immediate surroundings. The control algorithm tells each vehicle how to accelerate and decelerate.
Their system did not create stop-and-go traffic as vehicles approached the intersection. (Stop-and-go traffic occurs when cars are forced to come to a complete stop due to stopped traffic ahead). In the simulations, more cars went through a single green phase, which outperformed a model that simulates human drivers. Compared to other optimization methods also designed to avoid stops and starts, their technique resulted in greater fuel consumption and emission reductions. If every vehicle on the road is autonomous, their control system can reduce fuel consumption by 18% and carbon dioxide emissions by 25%, while increasing travel speed by 20%.
“A single intervention with a 20 to 25% reduction in fuel or emissions is truly incredible. But what I find interesting, and was really hoping to see, is this non-linear scaling. If we control only 25% of the vehicles, that gives us 50% of the benefits in terms of fuel and emissions savings. This means we don’t have to wait until we have 100% autonomous vehicles to reap the benefits of this approach,” she says.
Later, the researchers want to study the interaction effects between several intersections. They also plan to explore how different intersection configurations (number of lanes, signals, timings, etc.) can influence travel time, emissions and fuel consumption. Additionally, they intend to study how their control system could impact safety when autonomous vehicles and human drivers share the road. For example, even though self-driving vehicles may drive differently than human drivers, slower roads and roads with more consistent speeds could improve safety, Wu says.
Although this work is still in its early stages, Wu sees this approach as one that could be implemented more realistically in the short term.
“The objective of this work is to move the needle of sustainable mobility. We also want to dream, but these systems are big monsters of inertia. Identifying intervention points that are small changes in the system but have a significant impact is something that wakes me up in the morning,” she says.
This work was supported, in part, by the MIT-IBM Watson AI Lab.