[Documentation] [TitleIndex] [WordIndex

Note: This tutorial assumes that you can provide the /camera/image_rect and /camera/camera_info input topics described in the overview.
(!) Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Detection in a Video Stream

Description: This tutorial teaches you how to use this package for tag and tag bundle detection in a video stream.

Tutorial Level: BEGINNER

Next Tutorial: Bundle Calibration

Note: For the tag pose estimation to work properly, your camera_info topic must be appropriately set with your camera calibration parameters (see this Github issue).

The most common use case for this package is to detect tags in a continuous stream of images (video frames) coming from a camera. This tutorial teaches you how to do this.

Parameter Setup

config/settings.yaml

This file sets parameters for the core AprilTag 2 algorithm and for the ROS wrapper. Use it in order to set these parameters. In particular:

config/tags.yaml

This file defines the standalone tags and tag bundles that should be looked for in the input image. Rogue tags (i.e. those that are not defined in config/tags.yaml) will be ignored. IMPORTANT:

Furthermore, make sure that you print yours tags surrounded by at least a 1 bit wide white border. The core AprilTag 2 algorithm samples this surrounding white border for creating a light model over the tag surface so do not e.g. cut or print the tags out flush with their black border.

The tag_bundles are filled out with the relative poses of the constituent tags to the bundle origin. Sometimes you may know these transforms (e.g. you print the bundle on a single sheet of paper where you lay out the tags in a known fashion). When you do not know these relative poses (e.g. you stick individual tags roughly on the floor/wall), perform a calibration to easily determine them.

launch/continuous_detection.launch

The launch file provides an opportunity to set a few more parameters. Here, the camera name and image topic can be remapped to your implementation. Furthermore, you can set the publish_tag_detections_image=false in order to not publish the /tag_detections_image (more efficient if you do not care to see an image with the detected tags highlighted).

Run the Detector

With the parameters correctly configured and the camera image stream being published, running the detector is trivial:

$ roslaunch apriltags2_ros continuous_detection.launch

Example

Consider the following config/settings.yaml:

tag_family:        'tag36h11'
tag_border:        1 
tag_threads:       2 
tag_decimate:      1.0 
tag_blur:          0.0 
tag_refine_edges:  1 
tag_refine_decode: 0 
tag_refine_pose:   0 
tag_debug:         0 
publish_tf:        true

Consider the following config/tags.yaml:

standalone_tags:
  [
    {id: 10, size: 0.15},
    {id: 20, size: 0.1},
    {id: 30, size: 0.07}
  ]
tag_bundles:
  [
    {
      name: 'my_bundle',
      layout:
        [
          {id: 0, size: 0.05, x: 0.0000, y: 0.0000, z: 0.0, qw: 1.0, qx: 0.0, qy: 0.0, qz: 0.0},
          {id: 4, size: 0.05, x: 0.0548, y: -0.0522, z: 0.0, qw: 1.0, qx: 0.0, qy: 0.0, qz: 0.0},
          {id: 3, size: 0.05, x: -0.0580, y: -0.0553, z: 0.0, qw: 1.0, qx: 0.0, qy: 0.0, qz: 0.0},
          {id: 2, size: 0.05, x: 0.0543, y: 0.0603, z: 0.0, qw: 1.0, qx: 0.0, qy: 0.0, qz: 0.0},
          {id: 1, size: 0.05, x: -0.0582, y: 0.0573, z: 0.0, qw: 1.0, qx: 0.0, qy: 0.0, qz: 0.0}
        ]
    }
  ]

We start the camera (Dell XPS 13 9360 webcam with 640x480 [px] resolution), the detector, run rqt and view the /tag_detections_image and the detected tags' and tag bundles' poses in rviz (thanks to /tf):

With these settings, the detection runs at roughly 15-20 Hz on the Dell XPS 13 9360 laptop with a 640x480 [px] resolution video stream. According to Valgrind (Callgrind with KCachegrind), the performance bottleneck is the core AprilTag 2 algorithm.


2019-10-19 12:30