2021 IEEE Future Directions Workshop: The Future of Telepresence
1-3 November 2021
November 1: Talks by Experts - See speakers, titles & on-demand videos below.
November 2-3: Workshop Discussions (Invitation only limited to members of IEEE FD Steering Committee on Telepresence. If you are a domain expert or key stakeholder and wish to join, please contact This email address is being protected from spambots. You need JavaScript enabled to view it.).
“Telepresence is not science fiction. We could have a remote-controlled economy by the twenty-first century if we start planning right now” (Marvin Minsky, 1980)
Telepresence refers to a set of technologies that allow a person to feel as if they were present, to give the appearance of being present, or to have an effect, via telerobotics, at a place other than their true location. [Wikipedia]. Telepresence has been driven by the need for extending the natural human sensorimotor competence and enabling remote action and perception, in environments that are extremely difficult to reach for humans, or extremely dangerous to operate in, such as space, high radiation disaster zones, and micro and nanoscale environments. As one of the applications, telesurgery, and telerehabilitation, both under the umbrella of telemedicine, allow medical experts, to conduct a highly specialized medical operation at a remote location. COVID-19 has brought telemedicine to a new level that would perhaps take more years to obtain acceptance. Telemedicine is only an example, other forms of telepresence have also accelerated to face restrictions in travel and access to workplaces. Due to the revolutionary communication technologies which allow for ultra-low latency and ultra-high reliability, telepresence is making its impact in several sectors of human life, with a promise to reduce cost in offering a broad range of services, to meet societal needs, and to democratize access to various services (including medical services). It already allows remote operators to act as if present in a remote cabin/control room, pilots to operate drones, specialized workers to operate heavy equipment, to drive agricultural machines, or drive surface vehicles. While the horizon of telepresence is being extended due to the acceleration of medium technologies (such as communication, and robotics), new challenges are surfacing (such as security and agility) which would call for an active conversation and collaboration between several stakeholders, including leaders from academia, industry, medical and regulatory sectors.
This workshop provides a unique platform to initiate a focused conversation on various topics related to this emerging field of applied science. The objectives of this workshop are:
(1) to assess the current state of the art and practice of the telepresence field (including applications, technologies, challenges), and to forecast its advancement, and its expected impact to society, over the next decade, and (2) to assess the opportunity and benefit of forming an IEEE Telepresence Initiative, and prepare a White Paper to IEEE with findings and recommendations.
An IEEE Telepresence Initiative will reach a broad IEEE community, across many Societies and involving multiple geographic units. While certain Societies will find telepresence fitting with the core of their scope, among those, as examples, being Systems, Man, and Cybernetics, Robotics, and Automation, Control Systems, Computer, Computational Intelligence, Signal Processing, others will find that telepresence becoming increasingly popular in the applications, for example in IEEE Education Society, Engineering in Medicine and Biology Society, Industrial Applications, Intelligent transportation systems, Vehicular Technology, etc. One of the practical outcomes of the IEEE Telepresence Initiative would be to accelerate the missing technology components, create new interfaces for teleoperation, and encourage deployable integrated telepresence systems.
The workshop has two parts: a set of presentations by distinguished speakers, leaders in the field, and a discussion session, for domain experts and key stakeholders, aimed to generate a White Paper to IEEE Future Directions.
On-Demand Videos
The 2021 IEEE Future Directions Workshop: The Future of Telepresence sessions were recorded and are now available on-demand. Each session has been divided into separate videos for each speaker. Browse presentation list below and click on session titles to view the videos.
On-Demand Videos
The 2021 IEEE Future Directions workshop sessions were recorded and are now available on-demand. Each session has been divided into separate videos for each speaker. Browse presentation list below and click on session titles to view the videos.
Session List
Speaker biographies
Stephen Welby, IEEE
Introductory Remarks
Stephen Welby is the Executive Director and Chief Operating Officer of the Institute of Electrical and Electronics Engineers (IEEE). IEEE is the world’s largest technical professional organization dedicated to advancing technology for the benefit of humanity. IEEE and its members inspire a global community to innovate for a better tomorrow with nearly 400,000 members in over 160 countries, and through its highly cited publications, conferences, technology standards, and professional and educational activities. IEEE is the trusted “voice” for engineering, computing, and technology information around the globe.
Prior to joining IEEE, in 2015 Stephen was nominated by President Obama and confirmed by the US Senate as the Assistant Secretary of Defense for Research and Engineering. In this role, he served as the chief technology officer for the U.S. Department of Defense, leading one of the largest and most complex research, development, and engineering organizations in the world. He oversaw a $12.5B annual investment portfolio, managed internal and collaborative research and engineering efforts, drove a culture that valued innovation, and supported the department’s global technical engagement.
Stephen has more than three decades of government and industrial experience in technology and product development, including senior leadership positions at the Defense Advanced Research Projects Agency (DARPA). His technical experience includes development of leading edge aeronautical and space systems, robotics, machine learning, high-performance software, and sensor systems.
Stephen holds a bachelor of science degree in chemical engineering from The Cooper Union for the Advancement of Science and Art, a master’s degree in business administration from the Texas A&M University, and master’s degrees in computer science and applied mathematics from The Johns Hopkins University. He is a fellow of the IEEE.
Thomas Sheridan, Workshop Honorary Co-Chair, Professor Emeritus, Massachusetts Institute of Technology
Recollections of the Origins of Telepresence, and Some Suggestions
Professor Emeritus, Departments of Mechanical Engineering and Aeronautics/Astronautics MIT. BS Purdue Univ. MS Univ California Los Angeles, ScD MIT, Dr. (hon.). Delft Univ. of Technology, Netherlands.
Books: Man-Machine Systems (MIT Press 1974); Telerobotics, Automation, and Human Supervisory Control (MIT Press, 1992); Perspectives on the Human Controller (Erlbaum, 1997); Humans and Automation (Wiley, 2002); What is God (New Academia, 2014); Modeling of Human-System Interaction (Wiley, 2017); Respectful Atheism (Rowman, 2021).
Served as president, IEEE Systems, Man and Cybernetics Society, and Editor, Transactions on Man-Machine Systems. Received Norbert Wiener and Joseph Wohl awards, Centennial Medal and Third Millenium Medal, IEEE Fellow. Also Fellow Human Factors and Ergonomics Society: recipient Paul Fitts and Arnold Small Awards, Was president HFES. Received 1997 National Engineering Award, Amer. Assoc. of Engineering Societies; 1997 Oldenburger Medal ASME. Member National Academy of Engineering.
Susumu Tachi, Workshop Honorary Co-Chair, Professor Emeritus, University of Tokyo, Japan
Telexistence – Its 40-Year History and Future
Susumu Tachi is Professor Emeritus of The University of Tokyo, where he currently leads several research projects on telexistence, virtual reality and haptics, including the ACCEL Embodied Media Project at Tachi Laboratory, Institute of Gerontology. He is the Founding President of the Virtual Reality Society of Japan.
One of his earliest scientific achievements, shortly after his obtaining a Ph.D. from The University of Tokyo in 1973, was the invention of Guide Dog Robot (1976), an intelligent mobile robot system for the blind. This system was the first of its kind and came to be known as MELDOG.
In 1980, Dr. Tachi invented the concept of Telexistence, which enables a highly realistic sensation of existence in a remote place without any actual travel. Since then, he has been working on the realization of telexistence, and has completed TELESAR V (2012), an avatar robot system that enables the user to bind with an anthropomorphic robot and have a real-time sensation of being where the robot exists, and to feel the robot’s body as his or her alter ego through visual, auditory and haptic sensation.
Other achievements include Haptic Primary Colors (2007), Optical Camouflage (2003), and autostereoscopic VR displays such as TWISTER (2002), Repro3D (2010) and HaptoMIRAGE (2014).
From 1988 to 2018, he served as Chairman of the IMEKO Technical Committee on Measurement in Robotics and directed the organization of ISMCR symposia and received IMEKO Distinguished Service Award in 1997. He initiated and founded International Conference on Artificial Reality and Telexistence (ICAT) in 1991 and International-collegiate Virtual Reality Contest (IVRC) in 1993. He received the 2007 IEEE VR Career Award, and served as General Chair of IEEE Virtual Reality Conferences.
Paolo Fiorini, University of Verona, Italy
Haptics and Force Feedback in Telepresence
Paolo Fiorini received the Laurea degree in Electronic Engineering from the University of Padova, (Italy), the MSEE from the University of California at Irvine (USA), and the Ph.D. in ME from UCLA (USA). From 1977 to 1985 he worked for companies in Italy and in the USA developing microprocessor-based controllers for domestic appliances, automotive systems, and hydraulic actuators. From 1985 to 2000, he was with NASA Jet Propulsion Laboratory, California Institute of Technology, where he worked on autonomous and teleoperated systems for space experiments and exploration. In 2001 returned to Italy at the School of Science and Engineering of the University of Verona (Italy) where is currently Full Professor of Computer Science. In 2001 he founded the ALTAIR robotics laboratory to develop innovative robotic systems for space, medicine, and logistics. Research in these areas have been funded by several National and International projects, including the European Framework programs FP6, FP7, H2020 and ERC. His activities have been recognized by many awards, including the IEEE Fellow (2009), and NASA Technical Awards.
Mark W. Spong, University of Texas at Dallas
The Role of Passivity in Bilateral Teleoperations
Dr. Mark W. Spong is currently the Dean of the Erik Jonsson School of Engineering and Computer Science and Professor of Electrical Engineering at the University of Texas at Dallas. Dr. Spong received his D.Sc. degree in systems science and mathematics in 1981 from Washington University in St. Louis. He has held faculty positions at Lehigh University (1981-82), Cornell University (1982-84), and at the University of Illinois at Urbana-Champaign (1984-2008). At UTD, Dr. Spong holds both the Lars Magnus Ericsson Chair and the Excellence in Education Chair. Dr. Spong’s main research interests are in robotics, mechatronics, and nonlinear control theory. His notable awards include the 2016 Nyquist Prize from the ASME Dynamics and Control Division, the 2011 Pioneer Award from the IEEE Robotics and Automation Society, the first IROS Fumio Harashima Award for Innovative Technologies in 2007, the IEEE Transactions on Control Systems Technology Outstanding Paper Award, the Senior Scientist Research Award from the Alexander von Humboldt Foundation, the Distinguished Member Award from the IEEE Control Systems Society, the John R. Ragazzini and O. Hugo Schuck Awards from the American Automatic Control Council, and the IEEE Third Millennium Medal. Dr. Spong is Past President of the IEEE Control Systems Society and a Fellow of both the IEEE and IFAC. Dr. Spong’s work has been instrumental in establishing the theoretical foundations of robot control, and the results he has produced over the past three decades have been implemented in systems at companies and research development facilities around the world, including Sandia National Labs in New Mexico and the Jet Propulsion Laboratory in California. He has produced innovative solutions in robotics that have stood the test of time to become now-classic results in robotic control. His work also led to the first practical solution of the problem of time-delay compensation in bilateral teleoperation, which was a major impediment to the development of undersea and space robots, and he was the first to show how poor performance in robot arms due to uncertainties and joint elasticity could be overcome with advanced nonlinear feedback control methods that he helped to develop. Dr. Spong has also had a major impact on robotics education. He coauthored one of the most popular textbooks on robot dynamics and control, which is still in use after more than 20 years. In addition, he developed both hardware and software, marketed by a company he founded (Mechatronic Systems Inc.), which are being used by more than 200 universities around the world.
Jan van Erp, TNO and University of Twente, Netherlands
Embodiment as Key to ultimate Transparency in Telepresence
Jan van Erp is principal scientist with The Netherlands Organization for Applied Scientific Research (TNO) and full professor of tangible user interaction with the University of Twente. Jan obtained a master’s degree in Cognitive Science Leiden University and a PhD in Computer Science from Utrecht University, both in The Netherlands. His research focusses on multisensory perception and cognition, applied neuroscience, robotics, and human-machine collaboration in complex environments. Jan acted as program director for many R&D programs, and published more than a hundred peer reviewed papers (h-index 44). He serves on the editorial board of five scientific journals, and holds board and advisory function including (vice-)president of the Eurohaptics Society, general chair of Eurohaptics 2020, chair of the TNO Institutional Review Board for human subjects experiments, and chair of the NATO Research Group Cognitive Neuro-enhancement.
Allison Mariko Okamura, Stanford University
The Present and Future of Haptics for Telepresence
Allison Okamura received the BS degree from the University of California at Berkeley, and the MS and PhD degrees from Stanford University. She is Professor in the mechanical engineering department at Stanford University, with a courtesy appointment in computer science. She is an IEEE Fellow and is currently the co-general chair of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems and a deputy director of the Wu Tsai Stanford Neurosciences Institute. Her awards include the IEEE Engineering in Medicine and Biology Society Technical Achievement Award, IEEE Robotics and Automation Society Distinguished Service Award, and Duca Family University Fellow in Undergraduate Education. Her academic interests include haptics, teleoperation, virtual reality, medical robotics, soft robotics, rehabilitation, and education.
Günter Niemeyer, California Institute of Technology
Force Feedback and Transparency for Immersive Telerobotics: Is this the Path to the Best Systems
Günter Niemeyer currently holds a visiting appointment at the California Institute of Technology. Before the current appointment he was for 6 years a senior research scientist at Disney Research, Los Angeles, where he made robots more expressive, more capable, more interactive, and ultimately more entertaining. He received MS and PhD degrees from the Massachusetts Institute of Technology (MIT) in the areas of adaptive robot control and bilateral teleoperation, introducing the concept of wave variables. He also held a postdoctoral research position at MIT developing surgical robotics. In 1997, he joined Intuitive Surgical Inc., where he helped create the da Vinci Minimally Invasive Surgical System. He was a member of the Stanford faculty from 2001-2009, directing the Telerobotics Lab. From 2009-2012 he worked with the PR2 personal robot at Willow garage. His core research interests still focus on human-in-the-loop, force sensitivity and control, touch, and haptic interactions, believing a robot really is a robot because it physically manipulates the real world.
Leila Takayama, University of California, Santa Cruz
Human-Centered Explorations of Robotic Telepresence
Leila Takayama is a human-robot interaction researcher, who is an associate professor in the Baskin School of Engineering at UC Santa Cruz. With a background in Cognitive Science, Psychology, and Human-Computer Interaction, Dr. Takayama examines human encounters with telepresence and increasingly autonomous technologies. At Hoku Labs, she does consulting for tech companies and non-profit organizations, helping them to make more data-driven decisions about how to design autonomous products and services. Prior to founding Hoku Labs and joining the faculty at UC Santa Cruz, she was a senior user experience researcher at GoogleX, and was a research scientist and area manager for human-robot interaction at Willow Garage. She has served as a World Economic Forum Global Futures Council Member and Young Global Leader. In 2015, she was presented the IEEE Robotics & Automation Society Early Career Award. In 2012, she was named a TR35 winner and one of the 100 most creative people in business by Fast Company. Dr. Takayama received her BAs in Cognitive Science and Psychology from UC Berkeley, an MA in Communication from Stanford University, and a PhD in Communication at Stanford University.
Terry Fong, NASA
Space Telerobotics: Interactive Human-Robotic Exploration
Terry Fong is NASA’s Senior Scientist for Autonomous Systems and the deputy rover lead for NASA’s VIPER lunar rover mission. He is also Chief Roboticist and former Director of the Intelligent Robotics Group at the Ames Research Center. Terry previously led development of the Astrobee free-flying robot, which was installed on the Space Station in 2019. Terry has published more than 150 papers in space and field robotics, human-robot interaction, virtual reality, and planetary mapping. Terry received his B.S. and M.S. in Aeronautics and Astronautics from MIT and his Ph.D. in Robotics from Carnegie Mellon University.
Saeid Nahavandi, Deakin University, Australia
Haptically Enabled Tele-operations
Saeid Nahavandi is an Alfred Deakin Professor, Pro Vice-Chancellor (Defence Technology), Chair of Engineering, and the Director for the Institute for Intelligent Systems Research and Innovation at Deakin University. He is a specialist developer of modelling and simulation solutions for large complex systems (e.g. airport security. His research interests include modeling of complex systems, robotics and haptics. He has published over 900 journal and conference articles. Former Board Member at Simulation Australasia. He is a Fellow of Engineers Australia (FIEAust), the Institution of Engineering and Technology (FIET) and Fellow of IEEE (FIEEE). He is the Co Editor-in-Chief of the IEEE Systems Journal, Associate Editor of the IEEE/ASME Transactions on Mechatronics and an IEEE Access Editorial Board member. He received a Ph.D. from Durham University, U.K. in 1991.
John Blitch LTC, USA ret
Neurophysiological Assessment of Latency Induced Cognitive State in Space Based Emergency Response
John Blitch is a cognitive neuroscientist with an operational background in robot assisted emergency response, hostage rescue, special operations, and nuclear weapons delivery. As a recently retired senior scientist and private consultant, he endeavors to share misadventures and lessons learned with space-based first responders regarding the narrow overlap between these disparate fields in the hope of leveraging advanced technology development against the daunting challenges that lie ahead. He holds a Bachelor’s Degree in Civil Engineering from the U.S. Military Academy at West Point, a Master’s Degree in Math and Computer Science from the Colorado School of Mines, and an MSc and PhD in Cognitive Psychology from Colorado State University.
Carolyn J. Heinrich, Vanderbilt University
The Promise of Telepresence for Expanding Educational Opportunities
Carolyn J. Heinrich is Chair of the Leadership, Policy and Organizations Department, a Professor of Public Policy and Education in the Department of Leadership, Policy, and Organizations at the Peabody College and a Professor of Economics in the College of Arts and Sciences. Heinrich’s research focuses on education, workforce development, social welfare policy, program evaluation, and public management and performance management. She works directly with federal, state and local governments in her research to improve policy design and program effectiveness and also collaborates with nongovernmental organizations (such as the World Bank, UNICEF and others) to improve the impacts of economic and social investments in middle-income and developing countries. She received the David N. Kershaw Award for distinguished contributions to the field of public policy analysis and management in 2004 and was elected to the National Academy of Public Administration in 2011.
Carla Victoria Ramírez López, Tecnologico de Monterrey, Mexico
Telepresence in Education: Hologram Professors
Carla Ramírez is the Leader of Educational Innovation and the Hologram Professor Initiative at Tecnologico de Monterrey. Carla graduated with a bachelor’s degree in Computer Information Systems, and a master’s degree in Information Technology Management from Tecnologico de Monterrey.
She has more than 25 years of experience in the design and management of educational innovation projects, educational technology, user experience, development of learning spaces with new technologies.
Within the Tecnologico de Monterrey, she has worked as a Creative and Technological Design Team Leader, Web Portals and Multimedia Director, Instructional Design Coordinator, and Tutor in graduated online courses.
In 2020, the Hologram Professors project received three Reimagine Education Awards.
Michele Faragalli
Real-Time Control and Operations of Lunar Rovers
As Chief Technology Officer at Mission Control, Michele leads all technology development activities and R&D initiatives. Michele has broad engineering leadership experience, having led interdisciplinary engineering projects from concept to production across military, aeronautical and space applications. Michele’s research expertise is in robotic and planetary mobility. He is an adjunct professor in the Department of Mechanical & Aerospace Engineering at Carleton University. When not spending time with his young family, you can find Mic in the Gatineau hills on a pair of skis or pedaling hard on his bicycle.
Hari Nayar, NASA Jet Propulsion Laboratory, California Institute of Technology
The Next Phase of Space Exploration
Hari Nayar is the supervisor of the Robotics Surface Mobility Group and a principal technologist in the Mobility and Robotics Section at JPL. He has supported and led research tasks at JPL in the development of robotics technologies for surface mobility, manipulation, subsurface access, dynamics modeling & simulation, multi-agent systems, medical applications, and tele-robotics. He was a visiting lecturer in the Mechanical and Aerospace Engineering Department at UCLA. Hari received his BS, MS and ScD degrees in Mechanical Engineering from MIT.
Jeff Linnell, Formant
Remote Observability and Control of Field Deployed Robots
Jeff Linnell brings an unorthodox approach to robotics. Part cinematographer, part Creative Director and part self-taught engineer, Linnell, founded Ultraviolet (now UV Phactory), A digital design and production studio, and began a multidisciplinary career merging the three.
His work led him to San Francisco where he founded Autofuss and Bot & Dolly, a design/production company and an engineering studio specializing in automation, robotics and film.
Bot & Dolly developed proprietary software with the goal of making robotics accessible to humans, enabling artists in all forms to use them as a comfortable extension of their workflows. This culminated in filming the 2013 blockbuster, Gravity, as well as the short film Box, both of which garnered Bot & Dolly many awards and accolades. In addition to exploring the nexus of entertainment and technology, Bot & Dolly is well regarded as an automation leader in the emergent field of Digital Fabrication.
Both Bot & Dolly, and Autofuss were acquired by Google in 2013, and Jeff assumed roles focusing on human/machine interaction as well as VP Product at Google Robotics
Upon leaving Alphabet (Google) in 2016, Jeff along with key members of his design, engineering, and product teams formed Formant. Formant is dedicated to the enablement of next generation robotics through cloud connected software. Figure is focused on the aggregation, visualization, and sharing of ‘Robot Shaped Data’ to enable the next wave of automation applications.
David Locke, XPRIZE
Building an Avatar Future
As Executive Director of Prize Operations, David Locke brings nearly two decades of Operations and Management experience with a focus on producing large-scale, international competitions and events that drive technological breakthroughs and positive social impact. Over the course of his ten plus years at XPRIZE, David has orchestrated an eclectic slate of competitions with topics ranging from robotics, Moon exploration (Google Lunar XPRIZE) and oil spill cleanup technology and has awarded more than $75 Million US dollars in prize money.
David currently serves as the Prize Lead for the ANA Avatar XPRIZE, a $10 million, four-year global competition focused on the development of a robotic Avatar system that will transport a human’s senses, actions, and presence to a remote location in real time.
Adrian Stoica, Workshop Chair, NASA Jet Propulsion Laboratory, California Institute of Technology
Concluding Remarks
Adrian Stoica is Senior Research Scientist, Principal, and Manager, Strategic Programs, in the JPL Office of Strategic Planning. At JPL for over 25 years, he contributed as Principal Investigator on advanced technology projects, as Supervisor of the Robotic Systems Estimation, Decision, and Control Group (for more than 10 years), and in various programmatic roles. He obtained his MS degree from Technical University of Iasi, Romania, and his PhD from Victoria University, Melbourne, Australia. His contributions include pioneering work in learning by imitation of arm movements by humanoid robots, shadow biometrics, collaborative brain computer interfaces, and in evolvable hardware. He also developed concepts in space robotics, such as using reflectors to provide solar power to permanently shaded craters at the lunar poles. He is a NASA Innovative Advanced Concepts (NIAC) Fellow, and served twice as Vice-President of the IEEE SMC Society. In 2010 he founded Telepresence Mediators (Telemediators LLC), a company specializing in personalized telepresence services. He founded the NASA/ESA Conference on Adaptive Hardware and Systems in 1999. He is Chair for the 2022 NASA’s Breakthrough, Innovative, and Game-changing (BIG) Idea Challenge.
Organized by the IEEE FD Telepresence Steering Committee:
Adrian Stoica, JPL (Chair)
Günter Niemeyer, Caltech (Co-Chair)
Farokh Atashzar, NYU
Jeremy Bailenson, Stanford U
John Blitch, USA
Jan B F. van Erp, U. Twente, Netherlands
Michele Faragalli, Mission Control Space Services, Canada
Paolo Fiorini, U Verona, Italy
Terry Fong, NASA
Jeff Linnell, Formant
Tsutomu Hasegawa, Japan
Carolyn Jean Heinrich, Vanderbilt U.
Carla Victoria Ramirez Lopez, Tecnologico de Monterey, Mexico
Christine Miyachi, Nuance Communications
Alireza Mohammadi, U Michigan
Saeid Nahavandi, Deakin University, Australia
Hari Nayar, JPL
Allison Okamura, Stanford U.
Mark Spong, UT Dallas
Thomas Sheridan, MIT
Leila Takayama, U. C. Santa Cruz
Edward Tunstel, Motiv Space Systems, Inc.
Supported by:
IEEE Future Directions, IEEE Technical Activities
IEEE SMC Society, Technical Committee on Human and Robotic Space Exploration Systems
IEEE Robotics and Automation Society, Technical Committee on Telerobotics
IEEE Metropolitan Los Angeles Section, Joint Chapter of SMCS and RAS
Programmatic Support, IEEE Future Directions, IEEE Technical Activities
Mahjeda Ali, PMP, Project Manager, Future Directions
Kathy L. Grise, Senior Program Director, Future Directions
Christine Miyachi, Future Directions Committee Chair