Determining object geometry with compliance and simple sensors

—To determine object geometry in unstructured environments, sensors must be mechanically robust, must exert only low forces on objects during exploration, and must be able to scan large regions efﬁciently without risk of damaging objects or sensors. Joint-angle sensors on compliant joints provide an appealing option for this task. An algorithmic framework is presented that allows them to be used for contact detection and to determine object geometry without requiring tactile arrays or other complicated contact location sensors. This volumetric approach to using proprioceptive sensors provides improvements in accuracy over other existing approaches based on the intersection of planes and lines.


I. INTRODUCTION
Object geometry plays an important role in planning grasps and in characterizing and classifying objects.Unstructured environments such as human dwellings pose a particular challenge because they may contain unexpected obstacles, which frequently occlude machine vision systems and pose mechanical hazards to robot hands.Additionally, they often contain a wide range of objects that do not precisely match a priori models.Tactile sensing is particularly appropriate for determining object geometry and position, especially when combined with complementary sensors such as computer vision [1].Because they transduce mechanical parameters directly (such as surface contact and friction), tactile sensors provide essential information for mechanical tasks such as grasping.
The problem of determining object geometry from tactile sensors can be broadly divided into two parts: obtaining raw tactile data from an object, and using this data to create a model of the object.Tactile data can be acquired with a broad range of sensors.Although tactile sensors have been an active area of study for over 30 years [2], systems-level questions such as sensor size, complexity, cost, and surface geometry have limited their successful integration into robotic hands [3].For unstructured environments, completeness of coverage and mechanical robustness are key factors to avoid breaking sensors and disturbing objects.
Regardless of the transduction technology, the raw data these sensors generate most frequently consists of contact point locations, usually with surface normals [4]- [8].Some sensors also provide surface curvature [9], and Caselli et.al use bounding surfaces without contact location at all, though Fig. 1.Multiple exposures illustrating the use of compliant joints and joint-angle sensors to determine object geometry.Such a system is robust, inexpensive, and well-suited to the challenges of unstructured environments.
only for convex objects [5].This raw information is used to fit object models either for the local surface [10]- [12] or for the object as a whole [6], [10], [13], [14].Many such object fitting methods have been proposed, both by the grasp planning community and by the object classification community.Ultimately it is the application that determines which model is most appropriate-grasp planning requires accurate surface geometry, whereas classification requires differentiating features.A useful overview of object models is given in [15].
This work introduces a method to obtain raw tactile data using joint-angle sensors in compliant fingers over time.Such a sensing system is well-suited to the major challenges that exist in unstructured environments: it is mechanically robust to collision with objects and requires only basic positionbased control from the supporting arm to keep contact forces low while stroking an object.The localization algorithm is a space sweeping approach that generates a discretized enveloping surface as well as contact locations.It also returns the empty space around an object, which forms a natural basis for grasp planning in cluttered environments.Objects can be concave or convex, but are assumed to be static and unmoving.
In this paper, we present a method to detect contact using compliant joints, the Space-Sweeping method to localize contacts, and experimental validation.The advantages and limitations of this approach with respect to existing work is discussed.Finally, several promising directions for future work are noted.

II. CONTACT DETECTION WITH COMPLIANT JOINTS
To explore an unknown space with a finger, a sensor is required to detect contact with the target object and Fig. 2. Swept-Space Elimination.When contact is detected between a finger and a static object, the current finger surface defines a set of potential locations for this contact.If we assume the object is rigid and unmoving, the location of the contact cannot lie in any region that is subsequently occupied by the finger, and finger motion narrows the contact location.environmental obstacles.Ideally, such a sensor should detect the contact without applying large forces and should not significantly restrict the process of exploration through "blind spots" or velocity limits imposed by sensor fragility.
Many different sensors have been used for this purpose, including binary switches [13], tactile arrays [11], intrinsic force/torque sensors [16], piezoelectric films [17], piezoelectric resonators [4], and whiskers [18].Detecting contact with joint-angle sensors in compliant joints provides a number of advantages.The sensing area can be the entire finger link surface, and compliant joints do not generate high forces as a result of minor positioning changes.
The forces on such a compliant finger come from three primary sources: actuation, dynamics from the motion of the supporting hand, and contact forces with objects.The effects of actuation force and hand motion can be controlled or modeled, leaving any remaining deformation directly attributable to object contact.One method is to move the hand slowly with no finger actuation and apply a threshold to the joint deflection beyond which any deflection can be attributed to object contact.One artifact must be corrected-while the finger "snaps back" from extended deflection, the angle may still be past the threshold as the finger returns through free space.This can be corrected by a joint acceleration threshold.
When a finger thus equipped makes contact with an object, the force F exerted on the surface at the contact location x can be calculated from the joint stiffness K, the joint Jacobian J, and the angular deflection θ.For small deformations, this comes to Note that the force required to detect a contact is proportional to the joint stiffness and inversely proportional to the angular sensitivity of the joint-angle sensor and the radial distance to the sensor.

III. SPACE-SWEEPING ALGORITHM
As a finger moves through space, it carves out empty regions that do not contain the target object.This "sculpting" analogy inspires the following approach.

A. Assumptions
Assume that objects are rigid bodies that do not deform and do not move (i.e., static objects that do not experience large forces).Likewise, assume finger links are rigid bodies.Under these assumptions, the finger and the object cannot interpenetrate, and any space inside the finger cannot contain the object.
Assume a finger is instrumented with a sensor suite that serves two functions: to localize the surface of the finger in space, and to detect the existence of contact between the surface and the surrounding environment (a boolean condition).Also assume the existence of a control system that can move the finger through space while applying only minimal force to any object it encounters, e.g. a finger with compliant joints on a position-controlled hand.

B. Algorithm
Start with a region of interest in 3D space, i.e., a "target volume" that contains the object (identified, for example, by a computer vision system).This results in a bounding surface.
The quality of the localization depends on the ability to sweep the finger through voxels in the Potential Contact Set C that are in fact empty.Ultimately, this resolution is limited by the finger geometry and any surrounding obstacles that constrain permissible finger motion.For contact on a planar finger pad, the degree of localization is dependent on the local curvature of the object as shown in Fig. 3-sharp points can be narrowed to a single voxel, whereas flat areas can be only narrowed to a surface patch that provides an outer bound to object extent.The edges and corners of a finger surface suffer no such limitation and may be used localize flat and even concave regions provided the finger can fit inside the concavity.
To use this algorithm, a motion planner is required to narrow contact sets to contact locations.For smooth curved surfaces in two dimensions, it is sufficient to roll the finger against the surface.In three dimensions, it is additionally necessary to sweep it slightly from side to side to eliminate nodes containing the side surfaces of the finger.A more general approach that casts it as a motion-planning problem is presented in the discussion.

IV. EXPERIMENTAL VALIDATION
To validate the space-sweeping algorithm, the following experiment was performed.A finger was created consisting of two solid links joined by a compliant joint as shown in Fig. 4.Each link was instrumented with an electromagnetic tracker (miniBIRD, Ascension Technology Corporation, Burlington, VT) with a positioning resolution of ±0.5mm and an angular precision of 0.5 • .These read the position and orientation (represented as a 3x3 rotation matrix) at a rate of approximately 15Hz and store them to a file on a host machine.Together, they generate the information that would be received from robot forward kinematics and a joint-angle sensor on the finger.To eliminate arm control issues, the position of the finger was controlled manually by the experimenter: approach the object to contact, roll the finger against surface to generate rotation around the axis perpendicular to finger axis and normal surface.Contact was detected by measuring the magnitude of the angle between the distal and proximal trackers and thresholding it at 15 • .
The target volume was discretized as a quadtree of pixels to maximize the resolution in the regions of interest (at the surface of the object) and avoid wasting memory on empty space; in 3D an octree would serve a similar function.As the finger moves through the volume, regions containing the finger surface are divided to create high resolution in the region of the finger surface.The minimum size of such nodes was chosen to be 1x1mm in accordance with the resolution of the tracker.Contact sets were thresholded at a span of 4cm to be classified as contact locations, though in many cases (e.g.edges) the algorithm was able to localize contact to a single node as shown in Fig. 5.The algorithm was implemented in Matlab (R2010a-x86, the Mathworks, Natick, MA) and run on a personal computer, and the data structure used approximately 3000 nodes in each experiment.
The object was placed on a table in a calibrated coordinate frame, and the finger was stroked across its surface.Results are shown in Fig. 5.The algorithm was effective at determining object geometry, especially in regions of high curvature which are important for object recognition and classification.The error was measured by calculating the minimum difference between the actual object location and the center of the contact node, minus half the voxel width.For the block, this was 0.9 mm; for the cylinder it was 0.5 mm.

A. Summary
Using compliant joints and joint-angle sensors over time, it is possible to determine object geometry with a sensor suite that is simple, mechanically robust, and requirees only basic position control in the supporting hand.The Space-Sweeping algorithm developed here can be used to determine object geometry with this sensing suite under only the light assumption that the object does not deform or move.

B. Advantages
An understanding of the advantages of the algorithms presented here requires careful comparison with the extensive prior work on the problem of determining object geometry by touch.Single-sensor tactile probes consist of a single tactile sensor mounted to an end-effector, such as that used in [4].Although simple and potentially inexpensive, they require probing motions to avoid the possibility of contact with uninstrumented sections of the sensor ("blind spots") and precise control of the position of the supporting hand to avoid exerting large forces.Tactile Arrays can reduce the size of blind spots and potentially determine the surface normal and curvature as well as the contact location.However, they tend to be expensive and complicated, and popular commercially-available systems for robotic hands can cost tens of thousands of dollars.Deformable Fingertips filled with fluid have also been developed [19], but pose manufacturing challenges.Intrinsic force-torque sensors can be used to measure contact forces and locations over the full fingertip [7], but because they are fragile and expensive, they are better suited to manipulation than active exploration.Dynamic Sensors can measure small contact forces, but when used alone may completely miss the onset of contact with an object if it is too light, causing catastrophic failure [17].In contrast, joint-angle sensors on compliant joints are simple, robust, and inexpensive, although they do require motion to determine the object surface geometry.
Whiskers also use a passive compliant feeler to determine object geometry, but there are several important differences to the approach presented here.With whiskers, three primary approaches have been used to determine the location of the contact along the feeler.One is to measure the ratio of torque change to angular velocity τ / θ which can be used to determine the distance to a contact location along a compliant beam [20]- [24].This closely parallels the function of rat whiskers (vibrissae) [23].Lateral slip of the whisker along the surface of the object can bias readings and correcting it requires repeated measurements in multiple directions to correct [20].Implementing such an approach using a finger that is neither straight nor continuously flexible is not straightforward, so a secondary set of whiskers would be needed alongside fingers in grasping applications.Other approaches have used the changes in resonant frequency that result when a mechanically excited whisker touches an object [25].Using such an approach for grasping would also requires a separate set of whiskers due to the difference between continuous whiskers and robotic fingers that have only discrete compliant sections joining rigid links.
Finally, several groups have calculated contact location using the intersection point between whisker geometry at two different locations against an object, using rigid [26] or flexible members [27].However, this approach suffers from a fundamental geometric limitation: as two lines become closer to parallel, the location of their intersection becomes increasingly sensitive to noise in the sensor.This makes the approach unsuitable for large gently curved surfaces unlike the approach described in this paper.
Enveloping Grasps have been used for object classification since the early days of tactile sensing.Briot describes a Baysian classification approach in 1972 [29], and many others have used this sensing approach since then [5], [10].Although this approach also uses joint-angle sensors, it has two primary downsides: first, it does not capture concave features, which places limits on the object models that can Fig. 5. Experimental results for the Space-Sweeping algorithm used in two dimensions to trace a rectangle and a circle.The extent of the contact set is shown by the colorbar on the right-note that tracing with the fingertip would improve the localization on the straight edges of the rectangle.The maximum distance between the object edge and the edge of the contact node region was 0.9mm for the rectangle and 0.5mm and for the circle.
be used and the level of detail that can be expected.Second, executing an initial grasp requires sufficient prior knowledge of the object position and an unobstructed approach path to avoid collisions.This can be especially problematic in occluded environments where tactile sensing provides the greatest advantage over other sensing systems.
Self-Posture Changability (SPC) is another sensing approach to determine object geometry with joint-angle sensors [30] that was developed by Kaneko and Tanie.In SPC, the finger is moved incrementally over the surface of the object under mixed position control and active compliance so that the object surface causes a change in the position of the finger surface.Two finger positions can then be intersected to estimate the contact location.They also present analysis of the object geometry required to avoid the chance of deadlock when stroking fingers across the surface [31].
In contrast, the Space-Sweeping algorithm uses the finger surface itself as the measurement of the object surface, rather than the intersection of finger surfaces.This gives similar localization of the object in regions of high curvature (e.g.edges), but avoids the noise sensitivity generated by geometric intersection in regions of shallow curvature.
Joint-angle sensors are also used in Huber and Grupen's work [28].They use a Kalman filter to find the location of a hypothetical contact location on a multi-joint finger based on SPC posture intersections, the center of rotation of finger links, and the local surface velocity.The existence of contact is then validated using torque sensors to detect which link contacts are consistent.This enables the detection of fingertip contacts.Although they only develop the planar case, they Fig. 6.Compliant finger joints can also complement the approach presented by Huber and Grupen [28], obviating the need for impedance control and torque sensors.Contact is detected through joint deflection, and two criteria are used to narrow the location: at the contact, there can be no local surface velocity in the direction of the surface normal N ( x) • V ( x) = 0, and a single contact does not exert forces outside the friction cone

Object Geometry
note it is possible to extend to three dimensions under the assumption of a single contact.Implementing this algorithm with the compliant joint-angle sensors as shown in Fig. 6 would remove the need for impedance control to guide the finger along the surface of the object.
Compliant Joints and joint-angle sensors have also been used determine object geometry.Deckers, Dollar, and Howe present a conceptual framework casting contact localization on compliant fingers as a Markov decision process [32].Earlier work from Jentoft and Howe presents an initial geometric framework for using compliant joints and joint-angle sensors to determine contact geometry including fingertip tracing and contact localization, though only early experimental results are presented [33].More recently, Koonjul, Zeglin, and Pollard present three different approaches to localize contact points to one of 10 regions on the finger of a shadow hand: one based on torque equilibrium at the joints, an implementation of SPC using compliant fingers, and an empirical approach based on a classifier and training data [34].

C. Limitations
The most important limitation for using compliant fingers to detect contact comes from the slow speed required to avoid inertial effects generated by hand motion.This can be minimized by using lightweight fingers or by using a more sophisticated dynamic finger model.
For the Space-Sweeping algorithm, the most important limitation comes from the assumption the object does not move.This limits its suitability for use during manipulation when the object moves in a hand, though it may still work if the object is held fixed and stroked by a free finger.This premise also limits its use for very light or very compliant objects.The resolution is limited by the node size.This sets a lower bound on the precision that can be generated.However, this discretization error is isotropic and only causes a quantitative error (that scales with the node size), as opposed to the singularity created by line intersection approaches which may cause qualitative changes in the detected object geometry.More importantly, graspers can handle some amount of surface error-in some cases up to several centimeters [35]and below a certain scale, other parts of the system such as a robot arm controller create larger effects.Finally, the volume discretization requires more memory than a basic contactpoint representation.However, this is decreasingly important as the price of memory continues to drop.To reduce the memory needed, the discretization can also be applied locally on the scale of the finger and then converted to a more compact form once the surface has been determined.

D. Extensions
A number of attractive extensions exist for the Space-Sweeping algorithm.Surface normals can be added trivially to the algorithm's results by storing the finger state when contact is detected; if the contact location is narrowed to the surface of the finger, return the surface normal along with the contact nodes.This would reduce the effects of discretization, yielding a local enveloping polyhedral model of the object that would allow for concave features.
Another promising extension is to cast the motion decision more rigorously as a path-planning problem that would allow for the exploration of highly-obstructed geometry.The motion planner would need to prioritize regions of exploration according to a global goal, and choose motions to generate contact sets, narrow contact sets, and avoid catastrophic collision with the target object.All but the last item could be bundled into a cost function.The final item can be cast as a constraint in worldspace using the current finger state (location, orientation, and joint deformation), the maximum deformation resulting from motion into an unexplored area, contact force limits, and the existing spatial map already generated so far by the sensing algorithm.Note that these all result in geometric space constraints.A number of algorithms exist for such problems each with various strengths, including gradient descent, potential fields, rapidly-expanding random trees (RRTs), etc.

E. Applications
Grasp planning in unstructured environments is one key application for these algorithms.Grasping is a important skill for service robots in homes, hospitals, outdoor environments, and disaster regions, but significant uncertainty exists in these settings both with respect to the objects and the surrounding world.The low-cost and high mechanical robustness of the algorithms presented here match both the technical and the economic requirements for such applications.The algorithm is also useful for tactile mapping of the immediate environment because compliant fingers allow rapid exploration without danger of damage.Object classification is another important use.

F. Conclusion
Methods are presented to determine object geometry using compliant joints, joint-angle sensors, and hand motion.This sensor suite avoids the complexity and cost of tactile arrays and intrinsic sensors and significantly reduces the precision and accuracy of the control required for the supporting end-effector without affecting the quality of sensor data or generating high forces.
The Space-Sweeping algorithm determines a discretized bound for the object under very light assumptions about object geometry-the target object must be rigid and immovable under low forces, but it can be concave, irregular, and surrounded by obstructions.Because the method also determines the free space surrounding the object surface, it lends itself readily to grasp planning.

Fig. 3 .
Fig. 3. Contact localization depends on object curvature and the location of the finger contact.Using the finger pad, (a) Sharp corners can be narrowed more closely than (b) gentler curves.(c) Exploration with the finger pad can only establish an outer bound for concave regions, but (d) fingertip exploration does not have this limitation.

Fig. 4 .
Fig. 4. Experimental setup.The finger consists of two links joined by a compliant joint and fitted with electromagnetic trackers that are used to determine the location of the finger surface and determine the angle across the compliant joint.The base link is moved by hand, and contact is detected with a basic threshold on the magnitude of the joint deflection.