Mobile Manipulation for the Motor Impaired
Robots for humanity was a project aimed at developing mobile manipulator robot applications for improving the quality of life of people with motor impairments by collaborating with a potential user of this technology Henry Evans, an individual with quadriplegia. My contribution enabled Henry to scratch and shave his own face using a ROS-based system of controllers and behaviors for the PR2 robot. Force-torque sensing in the wrist and current-based collision detection in the joints allowed the robot to detect unsafe conditions and react safely to avoid applying any unsafe force. By fitting a 3D ellipsoid to Henry’s head, he could use latitude-longitude control commands to move the razor naturally around his face.
- KP Hawkins, PM Grice, TL Chen, CH King, CC Kemp, “Assistive mobile manipulation for self-care tasks around the head”, Symposium on Computational Intelligence in Robotic Rehabilitation & Assistive Technologies, 2014.
- KP Hawkins, CH King, TL Chen, CC Kemp, “Informing assistive robots with models of contact forces from able-bodied face wiping and shaving”, RO-MAN, 2012.
- TL Chen et. al., “Robots for humanity: using assistive robotics to empower people with disabilities”, Robotics & Automation Magazine, 2013.
- JM Beer, et. al., “Older users’ acceptance of an assistive robot: Attitudinal changes following brief exposure”, Gerontechnology, 2017.
- gt-ros-pkg/hrl-assistive/hrl_face_adl – Controllers/interfaces for manipulator movement around head
- gt-ros-pkg/hrl-assistive/hrl_head_registration – 3D perception for localizing head using face curvature
- gt-ros-pkg/hrl-assistive/hrl_ellipsoidal_control – Controller for moving tool around face using latitude/longitude controls
- gt-ros-pkg/hrl-sensor-utils/hrl_ft – Force/torque sensor utilities package
- gt-ros-pkg/hrl-pr2-behaviors/hrl_pr2_ar_servo – AR-tag-based feedback servoing controller for localizing the PR2’s base
- gt-ros-pkg/hrl-pr2-behaviors/hrl_pr2_arms – Python controllers for executing arm movement on the PR2
- gt-ros-pkg/hrl-pr2-behaviors/hrl_pr2_traj_playback – Utilities for recording backdriven arm motions and replaying them
- gt-ros-pkg/hrl/hrl_behaviors/hrl_behavior_manager – Global locking mechanism for robot resources
- gt-ros-pkg/hrl/hrl_behaviors/pr2_collision_monitor – PR2 collision detection suite for checking collisions with arms, force/torque sensors, and fingertips
- gt-ros-pkg/hrl/hrl_behaviors/pr2_grasp_behaviors – Simple, open loop grasping schemes for overhead and sideways grasping
- RethinkRobotics/baxter_pykdl – (Based on gt-ros-pkg/hrl-kdl/pykdl_utils) Python wrapper for Orocos’ Kinematics and Dynamics Library (KDL)
Human-Robot Collaborative Assembly
The focus of this research project was anticipating human behavior for safe and efficient human-robot collaboration in industrial environments. By tracking the human’s movement we can anticipate their future position and safely stop the robot when necessary. By tracking the hands of a person assembling one of several different constructions the robot can anticipate which bins will be needed next and deliver them without the human requesting them. The UR10 robot was mounted on a linear rail and both controlled by a real-time Linux kernel to achieve low-latency synchronized movement. I implemented ros-control drivers for both systems allowing for easy integration into the ROS system.
- KP Hawkins, S Bansal, NN Vo, AF Bobick, “Anticipating human actions for collaboration in the presence of task and sensor uncertainty”, ICRA, 2014.
- KP Hawkins, N Vo, S Bansal, AF Bobick, “Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration”, Humanoids, 2013.
- KP Hawkins, “Analytic inverse kinematics for the universal robots UR-5/UR-10 arms”, Technical Report, 2013.
- ros/urdf_parser_py – Python parser for URDF descriptions
- ros-industrial/universal_robot/ur_kinematics – Fast analytic inverse kinematics for the UR5/UR10 robots
- ros-industrial/universal_robot/ur_description – URDF description and visualization for the UR10
- ros-controls/rqt_controller_manager – GUI interface for the ros-control framework
- gt-ros-pkg/excel-util – ros-control-based control system for UR10 mounted on rail, ros C++ runs on host machine, C runs on UR-10 control box
- gt-ros-pkg/excel-util/indradrive_hw_iface – ros-control EtherCAT controller for the Bosch Indradrive control box
- gt-ros-pkg/excel-util/extrinsic_calibration – GTSAM bundle-adjustment utility for calibrating position/orientation of cameras using calibration board held by robot arm
Rationally Safe Human-Robot Systems
For applications like autonomous driving, agents need not be absolutely-safe, but rather rationally-safe, that is, safe under the assumption that the human will act rationally. One of the duties of a rational agent is that they will always attempt to avoid collision, up to a reasonable level of error in control and perception. Robots should be able to leverage these assumptions to make maneuvers, like pulling out in front of a human-driven vehicle, which force the human to engage in collision-avoiding behavior. I propose modeling the interaction as a differential-game robust-control problem. As potential time to collision decreases the robot more confidently assumes the human will act to avoid collision.
- KP Hawkins, P Tsiotras, “Anticipating Human Collision Avoidance Behavior for Safe Robot Reaction”, CDC, 2018.
- KP Hawkins, M Egerstedt, AF Bobick, “Towards rationally safe robots”, RSS Workshop, 2014.
Solving Stochastic Optimal Control Problems With Feynman-Kac Forward-Backward SDEs
The Feynman-Kac theorem from stochastic control theory suggests that the Hamilton-Jacobi equations associated with the value function of stochastic optimal control problem can be solved by a pair of forward and backward stochastic differential equations (FBSDEs). Further, application of Girsanov’s theorem tells us that we can choose the forward SDE at will so long as we compensate appropriately in the backward SDE. Utilizing this surprising result we can arrive at new methods for solving optimal control problems, such as using a rapidly-exploring random tree (RRT) to represent the forward SDE. FBSDE approaches could point towards a novel method for optimal control which combines the best aspects of differential dynamic programming (DDP), cross-entropy methods, and RRTs.
- KP Hawkins, A Pakniyat, P Tsiotras, “On the Time Discretization of the Feynman-Kac Forward-Backward Stochastic Differential Equations for Value Function Approximation”, arXiv preprint, 2021.
- KP Hawkins, A Pakniyat, E Theodorou, P Tsiotras, “Forward-Backward Rapidly-Exploring Random Trees for Stochastic Optimal Control”, arXiv preprint, 2021.