George Z.H. Zhu
Professor and Chair, Department of Mechanical Engineering,
York University
Research Interest: Dynamics and control of electrodynamic tether system for space debris removal, autonomous on-orbit service robot for space debris removal, and multifunctional materials for spacecraft structures.  

Speech Title：Path Planning and Visual Servo for Autonomous Free-Floating Robotic Manipulators by Reinforcement Learning

Abstract: This seminar will discuss optimal path planning and visual servo for a free-floating robotic manipulator to capture a non-cooperative target in space. A free-floating space robotic manipulator exhibits strong dynamic coupling between the arm and the base. The motion of the robotic arm disturbs the base spacecraft’s position and attitude, and the end effector’s pose depends on current joint angles and previous joint velocity. The robot interacts with the environment by visual serving, where an eye-in-hand camera is used to estimate and predict the target’s pose (position and attitude in 3D space). The motion estimation is based on either a combination of photogrammetry and an adaptive extended Kalman filter for better accuracy or neural network-based machine learning algorithms. Once the pose is estimated, a novel model-free path planning strategy is developed for a 6 DOF free-floating space manipulator using deep reinforcement learning to handle this challenge. The Deep Deterministic Policy Gradient optimization algorithm and the actor-critic algorithm are used to learn a policy that combines policy gradient and temporal-difference learning via trial and error in a simulation environment. A feedforward neural network with two hidden layers is employed in the process. The simplest form of the reward function, in addition to different constraints as opposed to more complex reward functions, is developed and evaluated. With the learned policy, the space robotic manipulator can schedule and perform actions for real-time applications. Computer simulations simulation is performed to validate the algorithms. A comparison of results using different reward functions and different mass ratios of manipulators over base spacecraft is discussed. The results show that the end effector can reach its target position with the required orientation, stay in that pose for a longer duration (for later capture), and use minimal joint motion. 

Brief Bio: Dr. George Zhu is a Professor and Tier 1 York Research Chair in Space Technology in the Department of Mechanical Engineering at York University in Toronto, Canada. He served as the inaugural Academic Director of Research Commons (2019-2022) in the Office of Vice-President of Research and Innovation. His research includes the dynamics and control of tethered spacecraft, autonomous space robotics, visual servo, CubeSat, and additive manufacturing in space. He has published over 350 papers in peer-reviewed journals and conference proceedings. Currently, he is an elected Corresponding Member of the International Academy of Astronautics; College Member of the Royal Society of Canada; Fellow of the Canadian Academy of Canada; Fellow of the Engineering Institute of Canada, Canadian Society of Mechanical Engineering, and Fellow of the American Society of Mechanical Engineers; and Associate fellow of American Institute of Aeronautics and Astronautics. Dr. Zhu is the receipt of the 2021 York University President’s Research Excellence Awards, the 2021 Robert W. Angus Medal from the Canadian Society for Mechanical Engineering, the 2019 Engineering Medal – R&D from the Professional Engineers Ontario, Canada.