Bringing humans and machines even closer with sensors and AI
Using BMI to operate machinery like your own limbs without any manipulation
As a result of the development of various machines and robots, we have become able to effortlessly handle jobs that humans could not perform with muscle strength and motor skills. Moreover, owing to advances in sensors, AI, and other information processing technologies, we have also become able to use perceptual abilities that exceed the five human senses. In order for humans to freely use the extended abilities realized through such machinery, other technologies that create a closer relationship between machinery and humans are required.
Scanning brain activity to create the ultimate HMI
When a human is operating a machine, the Human–Machine Interface (HMI) becomes the point of contact between both sides. Devices such as the steering wheel and pedals of an automobile, the keyboard and mouse of a PC, and the touch panel of a smartphone are able to operate a machine as an HMI. Recently, voice recognition technologies have progressed to the point where it has become possible to operate a machine such as an AI speaker using natural language as an HMI. Moreover, if you utilize the Virtual Reality (VR) technologies applied in games, etc., you can experience events within the virtual world created by the computer with a sense of immersion and feeling that you are actually there. The relationship between human and machine is steadily becoming closer due to the progress of technology.
However, with existing HMIs, a considerable degree of skill is required to communicate motions and operations you are thinking about to the machine. Freely operating a car as if it were part of your body like a racer or bus driver requires training that takes a lot of time. Moreover, when operating a machine through a tool such as a steering wheel, a slight delay inevitably occurs in the operation. To realize the kind of operation that feels as if human and machine have merged, technologies that further reduce the distance between both are needed.
So how much will the distance between human and machine be reduced in the future? The development of a technology to unconsciously move machinery like the limbs of your own body, which can be described as the ultimate form of HMI, is being advanced. That technology (Figure 1) is called "BMI" (Brain–Machine Interface). BMI has already reached the stage of practical implementation in some applications. This article introduces the trends in BMI evolution and the impact of this technology.
Visualizing what someone is thinking through advances in sensing and information processing
The development of sensing technologies to detect brain activity and information processing technologies to interpret (decode) the status of brain activity is becoming increasingly active. Thus, we are becoming able to realize an advanced BMI.
Until now, investigating brain activity required the use of large-scale medical equipment to measure brainwave changes and distribution within the head. Brainwaves are minute electrical signals that record from the top of the head, etc. the electrical potential occurring in the neuronal projections that make up the brain. To accurately measure brainwaves, you need to use 12 or more EEG recording devices to precisely measure electrical potential fluctuations of several microvolts.
Sensing technologies that accurately read brainwaves in a non-invasive manner without implementing any surgical interventions such as embedding electrodes in the head have advanced rapidly in recent years. Because brainwaves are observed as small voltage fluctuations called "microvolts," it was difficult to accurately measure them due to the impact of electrical noise generated by the muscles and electromagnetic waves in the surrounding environment. Owing to the development of technologies that remove noise, it has become possible to take highly precise brainwave measurements with miniature sensors.
In addition to the method of reading minute electric potential fluctuations, sensor miniaturization and increasing precision is advancing in methods that measure the magnetic fields generated due to brain activity and technologies that shine near-infrared light on the head to detect the states of light scattering and reflection to estimate changes in the volume of blood flow within the brain. Moreover, the use of a technology called fMRI (functional magnetic resonance imaging) equipment*1 to identify active areas of the brain with pinpoint high precision and measure the state of brain activity with greater detail is also advancing. By making effective use of these technologies according to different purposes, we have become able to know the state of brain activity from many angles.
*1 fMRI (functional magnetic resonance imaging) is a technology that measures the electromagnetic waves generated by the deoxygenated hemoglobin within the blood flowing in the brain to detect the activity of any location in the brain.
Moreover, technologies that interpret (decode) the biometric information that reflects brainwaves and other brain activity have also advanced, and we have become able to visualize cognitive information such as what a person is looking at or listening to. The technologies have developed to the point where today if you collect accurate brain activity information and effectively analyze it, you can even reproduce a fuzzy image on a monitor of the content of a dream that someone is having during sleep or images that a person is seeing with their eyes from brain activity information. Moreover, it is also possible to determine emotions and other mental states or the comprehension of studied material.
In recent years, it has become clear how the activity of brain cells and the various parts of the human brain relate to each other while operating by accurately modeling brain function based on deep-learning theory, which has become utilized in various fields as a core technology of artificial intelligence (AI), and the analysis precision has rapidly increased.
Wide range of BMI application fields, from entertainment to welfare and education
Because it has become comparatively easy to measure brain activity, attempts are being made to apply the information obtained through measurement to various applications.
For example, headphones that utilize brainwave sensors and are equipped with functions that can determine the degree to which a person is concentrating and in a relaxed state have been practically implemented. Games and other applications in which the character movement and story development change according to a person's mental state can be realized. When it comes to entertainment content, it is meaningless if the viewer or player becomes bored. However, no matter how interesting the content is, some people may find it boring. If you can provide content that triggers emotions by various means according to the mental state of each individual, then it will become something that will satisfy everyone.
In addition, there are also initiatives to utilize BMIs to support the lifestyles of those with disabilities through machines. Equipment that can distinguish between several hundred different types of messages from brainwaves have already been practically implemented, and it is hoped that it can be utilized as a means of communication for those with severe motor impairments that make speech and writing difficult. The arrival of an era in which prosthetic hands and legs can be manipulated like real limbs through the utilization of BMIs may not be far off.
Moreover, there are also initiatives underway to detect a person's brain response to received stimulation and apply that to impactful education and training to increase learning effectiveness. Systems that improve English listening capabilities in a short period of time by analyzing how English conversation audio is being heard and understood within the student's head and providing feedback are being developed.
BMI is an essential technology for minimizing the distance between human and machine as much as possible to merge both together. As the technology develops, it will become possible to freely manipulate massive and micro machines, machines placed in remote or dangerous locations, as well as computers with advanced computing power as if they were your own limbs. BMI will likely become a technology that extends human potential and supports the future.
- Continue reading:Super-Five Senses Created by Combining and Utilizing Sensors and AI