Autonomous vehicles are being tested both on open roads and in controlled environments. (Photo: Trimble)
The advent of autonomous vehicles (AVs) is one of three revolutions in the automotive industry that will likely change this country as much as cars did over the last century. The other two are the conversion from internal combustion engines to electric ones and the integration of cars into digital traffic networks.
Once mass deployed, AVs promise to dramatically reduce the number of traffic fatalities (42,000 in the United States in 2020, a National Safety Council report shows). They will never be sleepy, distracted, aggressive or drunk — nor will they engage in such inane human driving behaviors as texting while driving, playing chicken with bicyclists, or running red lights. They also promise to reduce fuel consumption, harmful emissions and traffic congestion by optimizing routes and increasing the number of people using car services instead of owning their own car.
To realize this vision, however, cars will have to do a lot more than just find their way on their own. They will have to perform flawlessly in an unpredictable world that includes toddlers, reckless drivers, fallen trees, sinkholes, construction and accidents.
See also Testing autonomous vehicles inside and out
GNSS Plus Corrections
Among the many sensors aboard an AV, the GNSS receiver has a unique role. It is the only one that can provide absolute positioning, in the form of latitude and longitude coordinates, to within a couple of decimeters anywhere on Earth. As such, it is “a key enabler to a lot of the vehicles to know precisely where they are and whether it is safe to activate autonomous systems,” says Gordon Heidinger, automotive segment manager, Autonomy and Positioning division at Hexagon.
A GNSS receiver cannot achieve the level of accuracy required for autonomous driving without robust corrections. Fifteen years ago, the state of the art was real-time kinematic (RTK) corrections. However, “the cost of that equipment exceeded the cost of a small car at that time,” recalled Steve Ruff, general manager, On-Road Autonomy Division at Trimble. “They were targeting a system cost of about $200. Today, that number is below $50, including the antenna, the GNSS positioning engine, and the software that runs on it.”
Today, all automotive manufacturers are using a form of precise point positioning (PPP) corrections, which is a one-way broadcast, as opposed to the two-way communication between a base station and a rover required for RTK. This means that a single correction stream can serve an entire continent, Ruff pointed out. “Once a vehicle is manufactured, we will support it with our PPP corrections stream for at least 10 years, which is the typical service life of a vehicle.”
Obstacles to Adoption
To achieve mass-market adoption, AVs will have to overcome numerous and complex obstacles:
- The technical difficulty of dealing with a limitless number of unanticipated challenges, such as poor visibility because of weather conditions, unpredictable human behaviors, complicated obstructions, detours and potholes
- The need to map millions of miles of roads, develop vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications, and protect vehicle software from hackers
- The difficulty, if not the impossibility, of handing off control to a human quickly enough to be safe when the system is unable to deal with a complex situation
- Questions about legal responsibility and insurance liability
- Ethical dilemmas about how to program the system to respond in emergencies
- The development of appropriate federal and state regulations
- Resistance from paid drivers who fear losing their jobs, including 3 million U.S. truckers, and from many other drivers, who fear losing control over their safety.
Trimble has approached all the major car manufacturers, has several programs in development, and has received multiple positioning requests for information (RFIs), Ruff said. “In 2018, Trimble’s RTX corrections service was the first solution adopted for production use in passenger vehicles, providing absolute precise positioning for General Motors’ Super Cruise system.”
Additionally, Trimble is working with Qualcomm and with SiriusXM, which will deliver Trimble’s RTX corrections over its satellite network, just like it does with music. “It is a good partnership because about 80% of the vehicles in North America are coming equipped with SiriusXM radio technology,” Ruff said. “The OEMs do not have to buy any additional hardware.” RTX corrections can also enter a vehicle via cellular IP, L-band satellite broadcasts and, potentially, over a V2I link.
Hexagon has proposed a PPP solution for automotive, “mainly because we essentially have the world covered with base stations, and that is a hard thing to do,” Heidinger said. “We have been running a corrections network for a very long time.” PPP’s one-way broadcast offers better cybersecurity because the GNSS receiver does not have to disclose its position, he added.
Swift Navigation is building a global corrections network. To make it suitable for the automotive market, the company is aiming to make its corrections service affordable and scalable. “We realized quickly that neither of the traditional RTK and PPP approaches were going to meet those requirements,” said Fergus Noble, company co-founder and CTO, “so we invested in developing a corrections service pretty much from the ground up.”
RTK provides high accuracy and short convergence times but is typically costly to deploy because it requires a very high density of stations, Fergus explained. As a consequence, most providers do not have continuous coverage over a wide area. Conversely, while PPP is a true global solution, it is less accurate and takes a long time to converge. “That may be fine in a marine or land-surveying application, but not if you are driving through city tunnels and bridges and need it to be able to reacquire a high-accuracy position within a matter of seconds. Therefore, we took a hybrid approach, together with a lot of new IP that we developed.” The service provides coverage in all the United States and most of Europe, and is being tested in Japan, South Korea and Australia.
Accuracy and Integrity
A common target accuracy for lane-level positioning is 20 cm 95% of the time. That means that AVs need to know when their positioning accuracy falls beneath that threshold. “We are building into our positioning solutions an accuracy metric that is output along with the position information we are providing,” Ruff said. “[The metric] can be used by the intelligence in the system to decide whether it can rely on the GNSS solution or needs to switch to one of the other complementary technologies because GNSS accuracy is not fulfilling its lane discipline.”
Heidinger noted the importance of economies of scale when mass-producing vehicles, where cost and ease of manufacturing become factors. “We can take some of our high-end equipment and get you 2 cm of accuracy with this technology, but the price point and the feasibility of this going into mass production for automotive is not favorable,” he said. “So, we’ve taken the approach of providing a software positioning engine that can be fit onto any hardware.”
Hexagon is developing products in partnership with STMicroelectronics, using the company’s Teseo V family of measurement engines. “ST is one of the established leaders of automotive GNSS solutions,” Heidinger said. “We take their measurements and put our positioning and corrections solution behind that to give positioning with lane-level accuracy.”
Noble agrees on the importance of knowing the reliability of a vehicle’s GNSS-based lane accuracy. The prevailing approach, which fuses data from GNSS and other sensors, makes it acceptable for one data source to be temporarily unavailable if the system is aware of that outage, he said. “That is where you start to see Swift, and others as well, focusing on the notion of integrity.”
An AV’s level of autonomy determines its behavior during GNSS outages. For systems with Level 2 autonomy and below, the driver must remain engaged, while Level 2+ and Level 3 systems will alert the driver to retake control when needed. If a driver of a Level 2+ or higher system fails to reengage, the AV’s reaction depends on the system and manufacturer.
“When we start to see Level 3 or above self-driving systems come onto the market, they will require that the GNSS component has an ISO 26262 safety certification,” Ruff said. “Many companies, including Trimble, are going through, or have gone through, the process of safety-certifying their offerings. As part of the AV system’s safety architecture, they will build in the capability to safely curb the vehicle if the system detects a malfunction or a spoof or some other type of problem.”
Automation Levels
In 2014, the international Society of Automotive Engineers released a standard, adopted in 2016 by the U.S. National Highway Traffic Safety Administration, that classifies cars in six levels, ranging from Level 0 (no automation) to Level 5 (full automation, meaning vehicles that can handle the full spectrum of road and traffic scenarios without any assistance from the driver). While many production models already incorporate various forms of Level 1 driver assistance, no current production car exceeds Level 2, or partial automation, which requires the driver to monitor the vehicle’s surroundings and take over as necessary. No test vehicle has yet achieved Level 5.
Image: GPS World
Other Sensors
Beyond lane-level positional accuracy, safe driving also requires avoiding collisions with other vehicles in the same lane or straying into it. Cameras, lidar and radar will detect other vehicles as well as fixed infrastructure and random obstacles, measure their distance, and monitor their movement.
While lidar scanners are better than cameras as detecting sharp-edged features, such as curbs, cameras are better at detecting and interpreting visual cues, such as road signs and the location and curvature of lane markers. In bad weather, radar is essential, because radio waves, unlike light waves, can penetrate rain, snow, fog and even dust, enabling radar to “see” where cameras and lidar cannot. However, radar sensors cannot see much detail, and cameras do not perform well in conditions with low light or glare.
Besides providing data about a vehicle’s trajectory, inertial navigation systems (INS) also measure its attitude (roll, pitch and yaw), enabling the software to better correlate and interpret data from the other sensors.
For example, when a car brakes sharply, its front end goes down; any forward-facing sensors measure distances to points closer to the car than they did a moment earlier, when its chassis was parallel to the street surface.
INS can also detect unsafe conditions, such as excessive slip angle, which is the angle between the direction of the rolling wheels and the vehicle’s true heading. A slip angle as small as 0.5 degrees can trigger skidding, spins or rollover, especially in the case of SUVs and tall trucks. Wheel-speed sensors also help verify the vehicle’s movement.
“All these technologies have their limitations,” Ruff said. “However, if you design the system, including all these technologies, then you can come up with a robust, safe combination that will enable autonomous driving.”
In addition to helping to avoid collisions, these other sensors provide relative positioning by comparing the images they acquire with highly precise maps to help locate the vehicle, especially in urban environments, which are well mapped and rich in recognizable landmarks.
Imagine an AV moving through different environments. It might travel from a city with urban canyons that degrade GNSS navigation, yet with landmarks that help relative positioning, to a rural environment devoid of both. The AVs’ algorithms must constantly weigh how much to rely on the different sensors. “Many of the OEMs and car companies are seeing that even rain mist on a highway is very bad for lidar and cameras, because it creates a big blur, but that is where GNSS will perform really well because it is open sky,” Heidinger said. “So, the two types of sensor systems complement each other very well.”
“Odometry sensors, such as a wheel-speed sensors, minimize any potential drift and add robustness to data that may have a GNSS outage of greater than 5 seconds, such as longer tunnels,” said Wesley Hulshof, principal engineer – ADAS Testing at Racelogic.
Photo: Racelogic
Noble sees a split in the industry. Companies such as Waymo and Cruise are pursuing Level 5 autonomy and are “heavy users of lidar” as well as other sensors. Companies such as Swift are focusing on Level 2 and Level 3 series production vehicles. “If you are making a mass manufactured vehicle for the production market, it rules out using a lidar sensor,” Noble said. “It is just too costly and complex right now to use. So, typically, if you look at the systems that are out on the market today, such as a Tesla Autopilot or a GM Super Cruise, they are very reliant on the camera as the primary sensor. Obviously, also inertial and some use of radar.”
Maps and Communications
While accurate and up-to-date maps have an important role to play in making autonomous driving possible, the more detailed maps are, the more the world they describe is constantly changing.
Meanwhile, the sensors keep improving and dropping in price, making maps less important. In the end, AVs — like human drivers — will probably rely much more on their ability to “see” and analyze their environment moment-to-moment.
Also like their human counterparts, they will gain experience. Unlike human drivers, however, AVs will be able to instantly share their experience with every other vehicle in their area via vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications.
V2V communications will enhance safety by informing AVs of the trajectories of nearby vehicles. If a vehicle is speeding toward an intersection and not slowing for a red light, it will be communicating its position and trajectory to other cars over a V2V link, Ruff explained.
“Then your car can make the intelligent decision to pump the brakes and avoid that collision. The same positioning stack that operates as part of the AV stack can also be used to support V2V-type applications, and the position of the vehicle will be much better than what the current V2V spec states.”
Different Approaches
Each GNSS manufacturer is taking a different approach to AV positioning.
The worlds of traditional automotive positioning and the products on which NovAtel has historically focused are coming together, Heidinger said. “The autonomous technology is demanding it and pushing for higher performance and safety-of-life functionality. Hexagon is bringing high-performance positioning solutions to the automotive industry in a manner that accepts automotive manufacturability, quality and efficiency.”
The company has also joined the 5G Automotive Association (5GAA), a large consortium developing AV solutions. “There are probably 100 companies in the industry coming together and helping to develop that vehicle-to-network communications solution, including telecom partners and automotive partners, and we are providing the GNSS expertise,” Heidinger said. “To meet the high-volume production-intent applications, including automotive quality, we recently developed a receiver based off the ST Teseo V family of measurement engines. We have an ST Teseo V set of chips on the PIM 222A product that launched in May geared exactly toward the automotive market.”
By contrast, Trimble is not focused on providing GNSS receivers or other hardware. “We allow the Tier 1 automotive manufacturers to architect the system using the components that they have selected from their preferred suppliers,” Ruff said. “We tailor our positioning solution to work with their architecture. So, we are agnostic as to the selection of the GNSS receiver, the IMU, the operating system running on the host system, and the host processor that runs the software. We can adapt our stack to run on virtually any system, using measurements from any GNSS source that meets our API requirements.”
For Swift, its “vision from day one has been to bring this type of precise positioning technology to mass market applications, such as automotive, which is a big focus for us,” Noble said. “That includes autonomy, but also ADAS, HD navigation and V2X. We do not want to be a hardware supplier in the automotive supply chain. Our boards are focused on professional and industrial markets.”
Swift’s automotive software, called Starling, runs on the vehicle’s computer. To generate a precise position, it ingests raw sensor data, as well as corrections data from the company’s Skylark network. “We focus on providing a precise-positioning stack that layers on top of any of this current generation of low-cost, automotive-grade receiver hardware from companies like STMicroelectronics.”
This test in London shows the value of inertial and wheel speed sensors. (Image: Racelogic)
The Future
Speculation abounds as to when AVs will enter mass production and how the transition from human to robotic drivers will take place. “There might be a ‘classics only’ lane in the future,” Heidinger said “that will be the only place where cars are allowed to be driven manually.”
Safety-enhancing automotive devices typically start out as optional extras, then get incorporated into best-practice standards promoted by independent bodies. Eventually, they become compulsory.
Some automakers have committed to creating their own AVs, while others are intent on creating a turnkey solution to transform conventional cars into driverless models. However, the initial market for AVs likely will be commercial fleets rather than individual consumers.
“It will still take quite a few years before we see cars take over and drive themselves, because legislation, insurance and these sorts of things will have to happen along with the technological advances,” Heidinger said. “But the positioning side is becoming more defined. We are seeing things like L5, the Galileo constellation, coming in and becoming more available. There are more constellations providing more data for use in our solutions, so that is promising.”
Swift’s Noble said, “Most of the major manufacturers working on Level 2+ and Level 3 systems are realizing that precision GNSS will be a key component of their architecture. Most of the major OEMs have signaled some level of intent to integrate this technology. Most are tracking to start the program next year,” he added.
“We envision that in five or six years every vehicle will have a single positioning utility on board that will serve all the location-aware applications on the car — whether it is an autonomous vehicle, V2V or V2I,” Ruff said. “It will meet the most stringent accuracy requirements from all the applications and serve navigation, telematics, security, V2X and AV/ADAS applications.”