At a recent all-hands meeting, Waymo’s chief winter weather expert explained it to all of the company’s engineers and product managers: If Waymo wants to expand into new cities and markets, its robotaxis will need to be able to handle snow confidently and safely.
During its early years in business, Waymo strategically focused on cities with hotter, drier climates, such as Phoenix, Los Angeles, Atlanta, and Austin. But as it eyes a string of East Coast cities, including Boston, New York and Washington, D.C., for the next phase of its expansion, its ability to handle more adverse weather will become a crucial test.
“Ha ha, good question,” said Robert Chen, the company’s weather product manager, when I asked him when he hoped to validate the Waymo driver for winter operation. “This winter season is going to be a really important season for us. I think that’s all I can probably say at this point.”
“This winter season is going to be a really important season for us. I think that’s all I can probably say at this point.”
It is understood that failure to cope with winter roads could hamper the usefulness of Waymo’s robotaxi service, which could limit its growth in new cities. Other human-powered rideshare services drive in all weather conditions. If it wants to compete, Waymo can’t just be a fair-weather robo-taxi company.
“I think here at Waymo we really aspired to create this product and service that people can rely on and use, not just eight, nine, 10 months out of the year,” Chen said, “but something that they can really rely on and use whenever they want.”
In some ways, autonomous vehicles are a lot like human drivers: They work best with good visibility, dry roads, and unobstructed sensors. When the roads become icy and snow accumulates, that’s when things get complicated. Waymo has faced a variety of challenges, from flash floods to Phoenix dust storms known as “haboobs,” but snow is a particularly tough challenge, he said.
image: Waymo
But human drivers can make assumptions about road markings and signage that can elude even the best-trained robot, said Phil Koopman, an autonomous vehicle technology expert at Carnegie Mellon University. Snow can obstruct road markings and traffic signs, making it more difficult for the vehicle’s perception system to respond safely and accurately.
“You may only see a third of the stop sign, but you know it’s a stop sign,” Koopman said. “Machine learning may have problems if it was not trained on partially obstructed stop signs.”
A system like Waymo’s, with its full stack of redundant hardware like lidar, radar and cameras, should eventually be up to the task, Koopman added. Systems using only cameras, like Tesla, might have more problems.
“It will be easier for a multi-sensor platform because the cameras will have a lot of trouble handling blowing snow,” he said. “But for sure, the radar will really help you if there’s snow.”
“Machine learning may have problems if it was not trained on partially obstructed stop signs.”
This is not only a technical challenge, but also a data challenge. Snowy conditions are rare in Waymo’s dataset, making relevant data extremely rare — often less than 5 percent or even a fraction of a percent of the total driving data for rarer conditions, Chen said.
This scarcity has pushed Waymo to use innovative techniques, including advanced AI methods, to augment and analyze data for development and validation, which Chen says is helping the company move forward. Waymo has logged some miles in snowy climates, including Truckee, California, Michigan, and upstate New York. Testing is also underway in Denver and Seattle, but there is still work to be done.
Waymo’s fifth-generation system can handle cold and light snow, but the sixth-generation Waymo Driver, expected to roll out soon, is specifically designed and tested to handle severe winter conditions, Chen said.
Image: Waymo
In addition to better data, Waymo is exploring other solutions to help its robotaxis navigate slipperier and muddier streets. For example, tiny mechanical windshield wipers were installed to help clear snow from the lidar sensor on the vehicle’s roof. More powerful heaters are also included to help defrost all sensors. Waymo’s current system is already suited to icy roads and can handle challenges such as icy conditions, Chen said, citing subzero temperatures in Austin last winter. Each vehicle in its fleet records this data while on the road, like a mobile weather station, he explained.
“Let’s say the vehicle encounters a slippery area,” Chen said. “It will actually send that information to the rest of the fleet and now other vehicles in the fleet will know that this particular spot is slippery.”
If conditions deteriorate to the point that roads become dangerous and most people stay home, Waymo may discontinue service. Such decisions are rare, but if the public isn’t traveling, there’s little need to keep vehicles on the road. And of course, every city is different: a light snowfall can paralyze some cities, while in others it would barely slow traffic.
This scarcity of data has pushed Waymo to use innovative techniques.
Even when the snow melts, Waymo can continue to expand its capabilities virtually, using advanced simulation models to replicate rare conditions. This is crucial to solving data shortage issues, Chen said. This is where AI comes in: generative and fundamental models are built into the system. And Waymo uses layered models that can distinguish between different types of snow – wet, powder, slush, etc. – and reinject this information into the training pipelines.
It may be a while before Waymo customers experience their first snowy robo-taxi rides. The company said it will begin operations in Washington, D.C., next year, but has not yet set a date for other East Coast cities. Waymo also aims to launch in London and Japan at a later date. But as the temperature drops and snow flurries begin to swirl in the air, Chen and his team will be ready.
“The autonomous driving problem… is really difficult in itself,” he said. “Now you have to add in these crazy weather conditions. It’s a pretty difficult task.”
