Main image of Technology to quantify an individual's mental state

Bringing humans and machines even closer with sensors and AI

Technology to quantify an individual's mental state

The human mind is an elusive thing. We often cannot determine what someone is really feeling, based simply on the words they speak and the expressions they show on their face.

However, some people have the ability to accurately read others based on their voice, behavior, and even slight changes in their facial expression. This includes psychologists, psychiatrists, and other people whose job is to read people's minds, but also includes expert salespeople and customer service staff in stores. Although the majority of salespeople and customer service representatives are not capable of performing at this level, there are some who continue to be successful due to their ability to make accurate proposals based on the latent desires of the customer. Being able to function at their level and accurately read the feelings and motives of others would dramatically increase work performance in all sorts of fields.

Image 1 of Technology to quantify an individual's mental state

It may be surprising to say so, but we are generally not fully cognizant of our own mental states. For example, you might make a mistake when making an important work-related decision if you are concerned about something or busy thinking about your plans for the weekend. We could avoid making these kinds of mistakes if only we could realize it when we are distracted. Professional athletes and individuals involved in creative work can experience drastic shifts in performance based on their own mental states. For people like this, being able to recognize and control their emotions can be the difference between success and failure. Taking an objective look at one's own metal state should be of help in developing the ability to stay calm.

Accommodating latent needs hidden deep within others

The development of so-called ""happy technology"" has become the subject of recent attention. Happy technology uses technologies such as sensors and artificial intelligence (AI) to quantitatively determine an individual's mental state, and then to suggest solutions to inspire better feelings. This kind of technology could be used when developing products and services to more effectively meet customer needs.

Image 2 of Technology to quantify an individual's mental state

Let's now take a look at an example of how happy technology could be used in product development.

When developing a product or service, a company will typically run statistics to determine customer needs based on previous results or marketing surveys, in order to determine what to develop. However, as consumers become more diverse and trends become more complicated, companies are now being required to develop products and services to satisfy increasingly individual and fleeting needs. In order to satisfy its customers, a company must thoroughly investigate its product and service development goals. Happy technology could provide companies with hints on the kinds of products and services it could develop to appeal to the individual values of customers with latent desires.

Surmising feelings from facial expressions, tone of voice, behavior, pulse rate, and more

The key to happy technology is technology to surmise feelings such as happiness, anger, sadness, euphoria, passion, and bewilderment. There are many factors that can be used to surmise an individual's mental state, such as facial expressions, body movements, casual behavior, voice, body temperature, perspiration, brain activity, and heart activity. A wide range of sensors can be used to gather information on these states or changes in state, which can then be fed into artificial intelligence (AI) or machine learning algorithms to guess how an individual feels.

The feelings that can be read and the accuracy at which they can be surmised varies by parameter, with each having strong and weak points.

For example, detecting feelings based on voice is more suited for surmising feelings not revealed by facial expressions, as the person being measured will have a harder time hiding their feelings (compared with how easily they might be able to control their facial expressions). This is because very few people can consciously control their vocal cords. In contrast, it is easier for people to control their facial expressions, and this tends to reduce the accuracy of such measurements. Having said that, a camera can be used to capture facial expressions from multiple people at the same time, and this can be useful in surmising the feelings of several people at once.

The ease at which data can be collected also varies from parameter to parameter. A camera or microphone can be used to gather data such as facial expressions, bodily movements, behaviors, and voices, without having to make any contact with the individual. In contrast, gathering data such as an individual's pulse rate, perspiration, and brain activity has generally required the use of sensors placed on the skin. However, advances have been made in technologies capable of gathering this kind of data without requiring any direct contact. For example, compact high-resolution millimeter-wave radars and ToF (Time of Flight) cameras capable of obtaining depth information can be used to detect an individual's pulse rate and breathing.

It would be even better if we could accurately and easily sense brainwaves and other kinds of brain activity, as this would allow us to guess how someone feels at the highest level of accuracy possible. Medical researchers in the clinical field continue to develop such technologies. However, we will need further breakthroughs if we are to reach the level where these technologies can be used in industry.

Use of this technology continues to expand in fields such as organization management and professional sports

Happy technology is already being used in a wide range of fields.

For example, the many sensors located in wearable devices and smartphones are being used to gather behavioral data (such as activity and pulse rate data), in order to guess how that individual feels. Such technology is being utilized now in managing teams in corporate organizations and business projects.

There are also many examples of videos captured by cameras being put to use. One example is a system that determines if a user is having trouble operating a ticket machine in a station based on the user's facial expression and behavior, so that station personnel can help. Cameras installed in sporting or event venues can capture footage of spectators and determine from their facial expressions whether they are enjoying themselves. These systems are even being used to adjust the timing of performances or sales based on the situation. Automobile manufacturers are now investigating systems that would use cameras to detect when a driver is not paying attention based on their facial expression and behavior. These systems could warn the driver if this occurs or could even automatically decelerate the vehicle.

Voice is also being used to determine an individual's mood. Technology with AI is now being used to analyze the frequency, rhythm, and speed of speech and to determine how an individual might be feeling. For example, this kind of technology was used with athletes on the Japanese team during the 2019 Rugby World Cup held in Japan, to determine their mental states. This technology makes decisions based solely on the characteristics of speech, without any understanding of what is actually being said. It can therefore be used to determine an individual's feelings from their voice alone, regardless of the language they are speaking.

Although they are currently limited in functionality, wearable devices for measuring the brainwaves of professional athletes are now available. These devices are capable of quantitatively measuring whether the right side of the brain (intuition) or the left side of the brain (logical thought) is more active. Researchers have shown that golfers tend to make more mistakes when the left side of the brain is more active. This technology is now being used to train professional golfers to gain an objective perspective on their own state of mind, so that they can better handle pressure during tournaments.

Incorporating functions to determine user feelings in home appliances and games

The use of happy technology could spread to unexpected fields in the future. Let's consider some interesting examples.

Some developers have proposed installing technology to read a user's mood in household products. For example, imagine an air conditioner or air purifier that could automatically adjust the room temperature or humidity, or that could automatically purify air, whenever it detects that you feel uncomfortable. Of course, there are already devices that will automatically adjust operation based on a set temperature. However, a temperature that feels fine for one person might be uncomfortable for another. AI could be used to learn an individual's preferences and adjust conditions to suit that individual. Eventually, this technology could be moved from individual appliances to the home itself, in order to monitor the moods of residents and automatically control conditions.

Image 3 of Technology to quantify an individual's mental state

There have also been attempts to develop video games that automatically adjust the level of difficulty or scenario based on the player's mood, in order to keep the player interested. If a game is too easy (or too hard), it could pull the player out of the game. The point when a player might start to feel bored when playing a game would depend on that individual's personality and how they feel at the moment. The concept here is to adjust the content for each individual in order to increase motivation. Some games being sold today already include this functionality on an experimental level.

Happy technology will bring innovation to various fields

Digital transformation (DX), in which data is put to full use in order to create new value in management, strategy, and business, is already being used in various fields. However, DX until now has only been able to make use of data that can easily be quantified such as business results and achievements, or physical or chemical amounts that can be directly measured with sensors. In business and others fields involving people, an individual's mental state can have a significant effect on results. Because this kind of data could not previously be handled as quantitative data, it could not be incorporated into DX practices.

However, as discussed above, we can now gather and analyze data on an individual's mental state. What's more, companies continue to prepare to include this data in their DX efforts. The use of happy technology has the potential to bring about astounding innovations in education, entertainment, politics, and other fields heavily impacted by feelings and mental states.

Other Links

Related articles