Chatbots powered by artificial intelligence (AI) algorithms, like ChatGPT, could have a major role in autonomous vehicles in understanding the commands given by drivers, according to a new study from Purdue University.
The study is said to be one of the first experiments in testing how autonomous vehicles can use large language models to interpret commands from a passenger and driver accordingly.
While many autonomous vehicles come with features that allow drivers and passengers to communicate with the users, they need to be clearer than would be necessary if you were talking to a human. By contrast, large language models can interpret and give responses in a more human-like way because they are trained from huge amounts of text data as well as keep learning over time.
“The conventional systems in our vehicles have a user interface design where you have to press buttons to convey what you want, or an audio recognition system that requires you to be very explicit when you speak so that your vehicle can understand you,” said Ziran Wang, an assistant professor in Purdue’s Lyles School of Civil and Construction Engineering. “But the power of large language models is that they can more naturally understand all kinds of things you say. I don’t think any other existing system can do that.”
Not driving but assisting
The large language models don’t drive autonomous vehicles but assist the vehicle in other features. Under the study, researchers found the vehicle could not just understand users better but would personalize its driving assistance to tailor to the person’s satisfaction.
Researchers trained ChatGPT with prompts ranging from direct commands — such as “please drive faster” — to more indirect commands—like “I feel motion sick right now.” Researchers gave the large language models parameters to follow that made it take into consideration:
- Traffic rules
- Road conditions
- Weather conditions
- Camera and ranging detection
Purdue University then made these large language models accessible over the cloud to an experimental vehicle with SAE Leve 4 autonomy — considered one step below fully autonomous vehicles.
In the experiment, the speech recognition system detected a command from a passenger and the large language model generated instructions for the vehicle’s drive-by-wire system — connected to the throttle, brakes, gears and steering — on how to drive according to the command.
Future directions
During the tests, the large language models average 1.6 seconds to process a command. The next steps are to improve this response time.
Additionally, these models tend to often misinterpret something they learned and respond in the wrong way. During the study, a fail-safe mechanism was installed to allow users to safely ride when the large language models misunderstood commands. This is another aspect researchers will work on.
While ChatGPT was used for many of the tests, researchers used other public and private chatbots based on large language models like Google’s Gemini and Meta’s Llama AI. ChatGPT performed the best on indicators, Purdue said. But that does not mean it will remain the best moving forward.
Another next step would be to see if these large language models of autonomous vehicles talked to each other, such as helping the vehicles navigate at a four-way stop. This project the team is working on as well as a test on how these models deal with extreme winter weather.
