Chilitags

I am the lead developer and maintainer of Chilitags, a software library extracted from code developed with my colleagues at Chili (formerly Craft). Chilitags is heavily based on OpenCV.

Fiducial Markers

Chilitags are 2D barcodes like QR-codes, except they are used as fiducial markers, while QR-codes are a way to transfer text (like a website address) via a camera. Fiducial markers like Chilitags are designed to be precisely located on an image. For example, Chilitags provide the (x,y) coordinates of their four corners with a sub-pixel precision.

Such fiducial markers can be used to build Augmented Reality systems (like Metroscope), by providing a computer with the position of objects in the real world, so that virtual information can be displayed on them. Typically, a 3D model of a virtual object can be placed on top of a piece of paper containing a tag. Robotics is another example application of fiducial markers: a camera embedded in a robot detects and tracks the positions of objects from the real world, or allows the robot to determine its position if the marker set-up is known.

There are other ways to recognize objects with a camera, e.g. with their natural, visual features, which avoid having to mark objects with an ugly barcode. Fiducial markers have their advantages though: it makes the user aware that the object is augmented, they are more reliable, and it is easier to define an object to the augmenting system, as only the numerical identifier of a tag has to be provided, versus complicated features.

Chilitags Features

  • Illumination tolerance: Chilitags were developed for use in projector/camera systems, which required the detection to be robust to the illumination constraints of the projector.
  • Precision: Chilitags were used in applications to teach geometry, which required the detection to be as precise as possible to match the requirement on the quality of work of the students.
  • Reliability: Chilitags were used for long experiments in ecologically valid conditions, which required to build systems stable even in various, uncontrolled settings.

Detection of Chilitags in unfavorable conditions

  • Low resolution: Chilitags can be recognized even from low resolution images (markers as narrow as 20 pixels are recognized). However, they are very sensitive to occlusions: if a border is occluded, the tag will not be detected.
  • Efficiency: Chilitags were run on Atom processors, which provide limited computing power.
  • Simplicity: Chilitags’s interface consists only in two classes, one for the detection of Chilitags, and one for the estimation of their 3D pose. They require little configuration and default to what is best in the general case, while allowing precise tweaking if needed.
  • 3D pose estimation: the 3D pose of Chilitags can be estimated easily for simple Augmented Reality applications. The calibration parameters of the camera can optionally be provided for an accurate 3D estimation.

The 3D estimation of a Chilitag in Javascript

  • Filtering: Chilitags embed mechanisms to paliate with the imperfections of the detection: flickering presence of tags and noise in the estimated coordinates.
  • Javascript version: an automated port (via LLVM) of Chilitags in Javascript is available.

Links