The most important attribute for a controller using visual-feedback is
having primitives which can accurately segment and track objects in
image space. Depending on the complexity of the experiment, this can
range from tracking a nut, to tracking the fingers of a robotic
hand, to tracking a complete robot.
A useful tracking primitive is the snake,
a model for representing image contours
which allows them to be easily manipulated by higher level processes.
The central idea behind a snake is that it is a deformable contour
that moves under a variety of image constraints (which tend to be
local) and object-model constraints.
The representation of a snake
is v(s) = (x(s),y(s)) where s runs from 0 to 1 over the
perimeter of the snake. The snake is controlled by minimizing a function
which converts high-level contour information like curvature and
discontinuities and low-level image information like edge gradients
and terminations into energies.
We have demonstrated the use of these snakes to control grasping with a monocular camera system and a pneumatic 4 fingered gripper from Toshiba. The goal of this experiment is to evaluate how well the snakes track when the finger moved over a sequence of image frames. The finger and the block were both estimated by a rough outline of a snake. Between successive images, the snake fitting routine was called a maximum of 20 times for each snake. The figure in the previous section shows snakes tracking positions of the fingers as they track a finger. Notice that the snake representation succinctly describes the pose of the finger. In addition, we have also verified that snakes can be used to detect contact between objects. Our collision detection algorithm reported a distance of 0 pixels between the two snakes (a point on the finger snake actually lies on a line segment contained by the block snake).
The next part of our experiment monitored the position of the finger-bolt-finger system while the fingers were translated along the robot's Z-axis. This part of the experiment is a simplified version of a general procedure for monitoring extractions. We verify the state of the extraction process by monitoring the positions of the bolt and fingers. If the bolt and fingers do not move in the same direction or if the fingers lose contact with the bolt, we say that the bolt is not grasped and the system must try again. Otherwise, we continue monitoring. On the previous html page, we showed the results of the extraction procedure given the initial starting state.