Two palms reconstructing the shape of an unknown object from tactile data.

Michael Erdmann and I have developed a method to reconstruct the shape of an unknown object using tactile sensors without requiring object immobilization. Instead, the robot manipulates the object without prehension. The robot infers the shape, motion and center of mass of the object based on the motion of the contact points as measured by tactile sensors. This allows for a natural, continuous interaction between manipulation and sensing. Eventually, it will make robots more capable in the physical world by enabling them to pick up unknown objects.

We have analyzed several different cases of the tactile shape reconstruction problem. First, we considered planar shapes with quasistatic dynamics. Simulations and experiments have validated the analytic results. Next, we extended the analysis to the full dynamics and prove observability of the nonlinear system describing the shape and motion of the object being manipulated. In our simulations, a simple observer based on Newton’s method for root finding can recover unknown shapes with almost negligible errors. Using the same framework we can also describe the shape and dynamics of three-dimensional objects. However, there are some fundamental differences between the planar and three-dimensional case, due to increased tangent dimensionality. Also, perfect global shape reconstruction is impossible in the 3D case, but it is almost trivial to obtain upper and lower bounds on the shape. The 3D shape reconstruction method has also been implemented and we present some simulation results.

tactile shape reconstruction in 2D

The picture on the right shows the basic idea of tactile shape reconstruction in 2D. The fingers rotate about the origin and sense the contact points. Based on the sensor data we can simultaneously reconstruct the shape and motion of the unknown object.

Below is a picture of the experimental setup. It is implemented using an Adept robot arm moving around the two palms (marked with the long white arrows). The object is also marked with an arrow. This is done so that we can sense the ‘ground truth’ with the Adept vision system and compare it with the shape and motion reconstruction from tactile data.

experimental setup

In three dimensions we need three palms:

The next movies show a simple motion of an unknown object rolling around on three stationary palms and the reconstructed curves traced out by the contact points. The convex hull of the curves provides a lower bound on the shape of the object.

  1. Mark Moll and Michael A. Erdmann. Reconstructing the Shape and Motion of Unknown Objects with Active Tactile Sensors. In Jean-Daniel Boissonnat, Joel Burdick, Ken Goldberg, and Seth Hutchinson, editors, Algorithmic Foundations of Robotics V, Springer Tracts in Advanced Robotics, pp. 293–310, Springer Verlag, 2004. [PDF]

  2. Mark Moll. Shape Reconstruction Using Active Tactile Sensors. Ph.D. Thesis, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, 2002. [PDF]