Prof. Dr. Gerd Hirzinger: Multi-sensor robotic systems to assist humans

“Following the light of the sun, we left the Old World.” Engraved in Christopher Columbus’s caravel “Santa Maria”, this quotation symbolises the human endeav­­­our to continue to face new challenges, to explore, and to discover – despite un­­known dangers. That said, humanity’s goals have long been elsewhere than in the discovery of new continents. Ironi­­cally, our object of desire lies only 100 kilometres over our heads and yet, it is considerably harder to reach. While Co­­lum­­bus had to deal with storms, waves and problems in provisioning for food and drinking water, the exploration of space poses completely different threats.

Already taking off into space is associated with large risks – and not just since the Challenger disaster of 1986. The sur­­vival of humans in the vacuum of space re­­quires massive safety precautions. Not only the enormous distances but also the radiation exposure and extreme temper­­­ature variations turn every voyage, albeit in a near-Earth orbit, into a perilous under­­­taking.


At the Robotics and Mechatronics Cen­­­­­­­t­er (RMC) of the German Aerospace Cen­­­t­­­er (DLR), headquartered in Ober­­pfaffen­­­hofen and Berlin, engineers, thus, re­­search how robots may be able to assume dangerous tasks. The goal is to de­­velop mobile ro­­bots that fly in Earth’s orbit or walk and drive on the surface of planets to assist humans. Those so-called “robonauts” are characterised by the aero­­nautic require­­ment of extreme weight and energy efficiency.

The abilities of human astronauts still by far surpass those of robots, not only in terms of cognitive adaptability to a large variety of situations but also due to the fact that human sensorimotor func­­­tions allow for much more detailed motion se­­quences. One of the key tasks of re­­search, thus, lies in providing the hands and arms of robots with gripping sensitivity.
The prototype models at DLR are equipped with numerous sensors for that purpose. The latest development is the “anthropo­mor­­phic hand-arm system”: The finger joints are individually moveable through a simulation of the tendons by way of two cables operated by miniature motors, based on the antagonistic drive principle of biological muscles. Thanks to the harmonious movement of both drives (agonist and antagonist), a finger seg­­ment is moved in the appropriate di­­rection. Since the drives operating in oppo­­site direc­­­tions via a special construction increases the tendon tension, the system is capable of increasing its stiffness by si­­­­multaneously contracting the antagonistic “muscles”, allowing it to adapt per­­fectly to a variety of tasks and environments.


With its six flexible metal wheels, the Exomars rover can overcome even large obstacles and inclines.

Modern industrial robots are capable of ex­­e­­­­cuting pre-programmed actions at very high speed, but are limited in their re­­spon­­­siveness. The more complex the motion se­quence is, the higher the planning and programming requirements in the prepa­­­r­­ation phase. However, research is advancing significantly in that area. Already, “an­­thro­­­pomorphic” (that is, human-like) robots of a whole new generation are being developed at the Robotics and Mechatronics Cent­er in Oberpfaffenhofen. With more than 40 joints, the two-armed, lightweight robot JUSTIN is able to react with excep­­tional sensitivity in interactions with hu­­mans. With its multi-sensor head, equipped with stereo cameras, laser scanner and light section projector, it can even recog­nise and grab a transparent glass, open bottles, pour beverages or lift heavy bottle crates. The main objective of this tech­­nological study is to demonstrate and fur­­ther develop the two-handed mani­­pulation of objects. Still, JUSTIN first has to ana­­lyse its environment and rigorously plan every single movement before it can act. While, through daily application, humans turn motion sequences into experience and routine, requiring less attention, JUSTIN must each time analyse his entire surround­­ings to be able to interact.


While JUSTIN represents the current pin­­nacle of humanoid robot development, the former DLR Institute can already look back on many years of experience with robot remote control in Earth’s orbit. As early as 1993, ROTEX, the first remote-controlled robot, was sent into space. From Tsu­­ku­­ba, the Japanese city of science, the Insti­­tute remotely programmed the first Japa­nese ETS VII robot to fly freely in space in 1999. From 2005 to 2010, DLR robotics special­­ists demonstrated the astronautic suita­bil­ity of its lightweight robot joints with the unique ROKVISS arm on the outer shell of the International Space Station (ISS), as well as the operation principle of the so-called telepresence concepts, which, by means of stereo image and force feedback, generate a feeling of on-site operation, often described as the extended hu­­man arm in space. Along with the Insti­­tute’s artificial four- and five-finger hands, the first astronautically suitable version of which is being used by the space agency ESA, those are all central building blocks for the European robonauts of the future.

RMC is further working on the development of the first European Mars rover, ExoMars, to fly to Mars in 2018, on rover concepts for moon landing missions, and on six-legged crawlers for so-called plane­tary exploration.
Furthermore, a ROBO­MOBIL derived from planetary rover tech­­­nology is to give new impetus to electric mobility while demonstrating the synergy between the fields of robotics and automotive technology. It is designed to be operated independently by means of an autonomy concept based sole­­ly on cameras, but also by a driver with a sidestick or remotely by a teleoperator.


In the meantime, multiphysical modelling, dynamic simulation, and control engineer­­ing applications of mechatronic systems are also increasingly benefiting classic terrestrial automotive technology, such as in the optimisation of driving performance and the development of components. The dynamics and energy efficiency of large aeroplanes are being modelled and opti­­mised in a similar way (in cooperation with Airbus). While the Center’s manned flight robots are generating 3D building models with their cameras and are already preparing for “manipulation from the air”, the solar-powered high-altitude platform ELHASPA is to rise into the stratosphere in the near future. Meanwhile, the Berlin location of RMC is working on developing small satellites, in particular, for detecting
fires on Earth. They are also supporting work in the field of photo-realistic 3D world modelling (interactive landscapes and mon­u­­ments) by developing innovative camera techno­lo­gies and multispectral sensors suit­­able for applications on planetary probes as well as satellites and aeroplanes. The Semi-Global Matching SGM is now count­­ed among the best stereo algorithms, not only used to create 3D models of moun­­tains, such as the Mount Everest and the K2, photographed by satellites, but also used by Daimler-brand cars in realtime obstacle re­cog­­nition.

Last but not least, MIRO, the robot developed from lightweight techno­­logy, represents a breakthrough in sur­­gi­­cal robotics. With a net weight of merely ten kilogrammes and compact measurements resembling a human arm, it can be employed right be­­side the surgeon at the operating table, where space availability is limited. The planned application possibilities of the robotic arm range from manoeuvring a laser unit to drilling holes for bone screws to minimally invasive “keyhole” surgery. Technologies from the Institute also find application in our daily lives. For example, the so-called SpaceMouse, which was developed as a requirement for the spatial control of DLR robots, was later turned into the world’s most popular 3D human machine interface under licence of computer equipment man­u­­facturer Logitech (with more than one mil­­lion installed in 3D computer graphics and construction systems).


By creating and securing over 1,000 high­­­ly specialised jobs in the industry and earn­­ing numerous national and international awards, the DLR Institute in Ober­pfaffen­hofen has gained the reputation of an inter­­nationally renowned technology centre and largely contributed to the joint decision by the federal and Bavarian state governments to expand it to the Robotics and Mecha­­tronics Center (RMC), one of the world’s larg­est facilities in applied ro­­botics re­­search. Where the best resources converge, the realisation of great goals and the discovery of new worlds is merely a matter of time.


1HirzingerFotoThe author is professor of robotics and, since 1992, he has been head of the Ro­­­botics and Mechatronics Center (RMC) of the German Aerospace Center (DLR) in Oberpfaffenhofen. He has further written over 600 scientific publications and earned numerous awards, including the Order of Merit of the Federal Republic of Germany and the Gottfried Wilhelm Leibniz Prize.