Controlling unmanned aerial vehicles (UAVs) is a challenge that has become increasingly important over the past few years, especially given their growing importance across a variety of applications, from surveying and delivery, to infrastructure inspection and warfare.
However, the vast majority of UAV accidents, which occur at a rate of about 24 per 100,000 hours of flight time, happen because of human error. Various labs are working on potential technological solutions to reduce that rate. One particular technology is not only capturing imaginations, but also brainwaves.
Brain-computer interfaces, or BCIs, have been under development in labs for decades. They range from rather invasive, where a user has an electrode directly implanted into their brain, to noninvasive, where the user simply wears a cap that looks like a prop from a sci-fi movie. Those caps contain electrodes, which can measure the five types of brain waves through a user's skull. A computer then uses training data and advanced learning algorithms to translate those brain waves into action in the real world.
Applications are wide ranging. Less-abled people could move objects in a room for the first time in years, just with their thoughts converted to instructions for a machine. Workers could easily manipulate robotic arms as easily as their own hands. And for something like a UAV, aircraft control becomes innate, with relatively less training and less risk of user error.
Researchers, such as Zhuming Bi of Purdue and his co-authors, have seen the potential of BCIs to correct human-introduced errors while piloting drones. A new paper discusses several types of errors and details how BCIs can help eliminate them. One of the most common types of errors, is input that is given to the drone interface incorrectly. BCIs could also potentially be used to reign in rogue or unstable pilots, which is another challenge altogether.
How BCIs can enhance drone control
First, let's look at interface corrections. UAV interfaces vary, with some looking like traditional aircraft controls, with a joystick and accelerator, while others resemble a hobby RC vehicle or video game controller. Each interface requires specific actions from a user to control the drone correctly, and one wrong move, such as an inadvertent twist of the joystick or accidental bump of a button on a controller, could have catastrophic consequences for an expensive aircraft that might be unrecoverable.
This is where a BCI could help. Tying intended motions directly to brain patterns can tell a UAV controller to ignore any input not directly tied to a brain pattern intended to provide a command. There is some concern about false positives when using such an interface. Still, with sufficient training data, most learning algorithms can differentiate intended brain states versus accidental ones, potentially saving expensive aircraft systems and recovery efforts.
The other use case discussed in the paper is undoubtedly more controversial. It examines whether a BCI could detect whether a human was in the right emotional state to control the drone. If the BCI and its associated algorithm decided the human pilot was not — say they were actively upset, stressed or angry — in control, some functions of the UAV would be ceded to the drone's software.
Interfacing those two control methodologies — the human and the software — is difficult. In the paper, the authors describe the development of software that would include a learning algorithm that analyzes the human's affections and another one that arbitrates between what the human wants to do and what the computer-controlled algorithm suggests.
While that might sound concerning, Dr. Bi and his colleagues are not alone in that effort, as they point out several research papers that have looked at AI algorithms whose sole purpose is to decide whether a human or a computer takes control of things in certain situations, even outside of direct drone control. This is similar to an ADAS system that senses a sleeping driver and stops the car.
An impressive UAV BCI example emerges
While the platform and its associated control schemas have a long way to go for development, a team from China recently published a paper in Nature Electronics that described an interesting use case of a BCI to control a UAV directly. Their system used a memristor chip comprising over a hundred thousand electronic components that can "remember" what voltage was applied to them by changing their resistance. This type of electrical feature makes them invaluable for implementation in a neuromorphic system, precisely the type of system the authors wanted to implement.
The results were impressive. Test subjects successfully navigated a drone through an obstacle course by simply thinking how they wanted a drone to move in a specific direction. The EEG helmet they were wearing picked up those brain waves, translated them into intent, and then moved the drone in the intended direction. Another learning algorithm continually updated itself with new information about the user's brain signals, increasing the accuracy of its estimations of their intent.
Memristors help stabilize some of the variability associated with operating off dynamic brain signals. They hold on to previous knowledge of what signals the brain was outputting but don't have to store it directly in computer memory, making them 1,000 times more efficient for this type of application.
That efficiency is one reason the system implemented a "closed loop" feedback system, where the "decoder" used to determine what the user's brain signals were trying to do was constantly updated based on their intent. As it was updated, the user's brain figured out how to provide clearer input into the decoder through the user's natural learning and feedback mechanisms. The result was an efficient system that could form the basis of effective UAV control using only a person's brain.
Out of the lab, into the air
Controlling drones is increasingly critical, but human error remains a leading cause of UAV accidents. BCIs offer a promising solution, translating brainwaves into commands to reduce errors and streamline control.
BCIs could prevent unintended inputs and even assess a pilot’s emotional state, temporarily handing control to AI when necessary. Recent breakthroughs, like a Chinese team’s use of memristors and closed-loop feedback to steer drones with thought alone, show real promise.
While still experimental, BCIs may soon revolutionize UAV operation across military, commercial, industrial and civilian domains alike.
