Main image of Haptics, and reaching a state of "unity of operator and machine"

Bringing humans and machines even closer with sensors and AI

Haptics, and reaching a state of "unity of operator and machine"

Sight, hearing, smell, taste, and touch. These five senses are crucial for the survival of all living things. An organism's five senses are tuned to the environment or ecology in which it lives, and keener senses allow for heightened decision-making and a better ability to translate thought into action. This is particularly true for humankind. We make full use of these senses at work and elsewhere. An office worker uses his sense of sight to read documents on a computer screen. A salesperson uses her sense of hearing to listen to a client, and her sense of sight to read facial expressions. A chef uses all five senses to create a delicious meal.

Professionals and "unity of operator and machine"

It's said that the difference between humans and machines is that we use our five senses to understand our surrounding environment and circumstances. Of course, there are machines fitted with specialized sensors used to perform certain tasks very accurately. However, even these machines are limited in terms of what kinds of tasks they can perform. A cutting-edge industrial robot equipped with advanced sensors will perform as instructed but can have trouble working alongside unpredictable humans. The relationship between humans and machines has therefore always been one of a superior over a subordinate—or of an operator and his tool.

Some highly skilled professionals have truly mastered this relationship and seem to operate their machines as an extension of themselves. An expert machinist using a lathe or milling cutter senses subtle changes in vibration and feel to produce astoundingly precise metalwork. Similarly, a professional race car driver senses and reacts to changes in gravitational acceleration and vibration from the road surface when cornering to save precious seconds of time.

Image 1 of Haptics, and reaching a state of "unity of operator and machine"

In horseback riding, an expert rider seems to work in perfect harmony with his horse. We might describe this as a "unity of rider and horse." Similarly, we could describe the relationship between a professional and his machine as a "unity of operator and machine."

Modern society and our tendency to rely on visual and auditory information

What, then, is the difference between a regular person operating a machine and a professional who is capable of truly amazing work? It may be a person's ability to make full use of his five senses.

Our homes and workplaces are now filled with smartphones, PCs, and other digital tools. Due to the prevalence of these tools, we now tend to focus much more on visual and auditory information when making decisions and translating thought into action. Of course, it's only natural for us to behave in this way. Digital tools bring tremendous convenience to life and work. However, these tools cannot provide information that appeals to our senses of smell, taste, and touch. Somehow, we all became accustomed to this.

It is stated in "Industrial Training System Handbook" (edited by the Training Device Editorial Committee, and published by JUSE Press) that 83% of the information we sense is obtained through our sense of sight, and 11% through our sense of hearing. That means only 6% of what we sense is obtained through our other senses (3.5% for smell, 1% for taste, and 1.5% for touch). Because our senses of sight and hearing account for over 90% of the information we receive, you might think it's fine to rely mostly on just these two senses. Many people would probably agree with you. However, we will need to make use of all of our senses if we want to make better decisions and perform at a higher level. Indeed, this might be the key to bringing human-machine interfaces (HMI) to the next level—and bringing humans and machines even closer.

Image 2 of Haptics, and reaching a state of "unity of operator and machine"

Haptics used to provide a sense of touch for digital tools

Thanks to advances in sensors and actuators, as well as technologies such as ergonomics, it is now possible to reproduce haptic sensations*1 using digital technology—and to use this as a means of reproducing information. This is the technology of haptics.

*1 "Haptic sensations" refers to many things. This general term is used to refer to a variety of sensations that can be felt on the skin, including the sense of force (sensation of being pulled or pushed), the sense of pressure (the sensation of touching something and feeling that it is hard or soft), and the sense of touch (the sensation of an object's surface texture).

When used as an HMI, haptic technology can communicate information to the user that could not be detected by sight or sound alone. Haptic sensation is what one first feels when touching something. Haptic sensation is also what allows us to determine that we are touching an actual object. We can use this human trait to help communicate information on substance and feel that could not be obtained through sight and sound alone. Haptic technology holds great promise as a means of bringing a greater sense of reality to games and virtual reality.

Using digital haptic information to bring humans and machines together

Haptic technology isn't just for games and VR. Digital haptic information could be used to create electronic devices and tools with novel functions. Information can generally be processed and manipulated easier in the digital realm. For example, visual information such as text, figures, and images can be recorded and viewed even as analog data. Converting this into digital information opens the door to a wide range of possibilities, such as translating Japanese text information into English.

Similarly, haptic information could be converted into digital information. Let's consider some thought-provoking examples.

Imagine someone operating a robotic arm to pick up and move objects. Haptic technology could be used here to provide the operator with a sense of touch, to better handle delicate objects or things with complex shapes. Now imagine that the operator's movements could be attenuated, while the robotic arm's sense of touch and feel could be amplified and provided to the operator as haptic sensation. This robotic arm could be used for surgery, with the operator actually able to feel what they are touching. Now imagine the opposite scenario, where the operator's movements are amplified while the feeling of weight of the object carried by the robotic arm is attenuated. An operator of a crane at a construction site could use this to control the machine as though it were an extension of their body. These are just some examples of how humans and machines could truly act as one.

Image 3 of Haptics, and reaching a state of "unity of operator and machine"

These examples are not science fiction, but describe actual technologies being researched and developed today. Haptic technology will become crucial in allowing humans and machines to work as one, and for us to control these machines as extensions of our bodies.

Robots are on the verge of seeing, hearing, smelling, tasting, and touching just like us

We've already discussed how we can use haptic technology to operate machines with a higher degree of flexibility and precision. You may have also heard that electronic control circuits and artificial intelligence (AI) can be used to operate machines. These technologies will ultimately allow us to create autonomous robots capable of making advanced decisions and performing high-level work based on haptic sensation, just like an expert machinist, professional race car driver, or doctor.

We have long been able to use digital technology to handle visual and auditory information. It is now possible to handle tactile information using this technology. Research has also made progress on using digital technology to handle smell and taste information, such as smell sensors that analyze a patient's breath to detect illness, or taste sensors that reference taste databases to set flavors. Researchers have only just begun developing technologies that detect information from all five senses and convert it into digital information. This holds enormous potential, and could very well redefine our relationships with machines.

Related articles