The Event-Camera Dataset and Simulator

This presents the world's first collection of datasets with an event-based camera for high-speed robotics. The data also include intensity images, inertial measurements, and ground truth from a motion-capture system. An event-based camera is a revolutionary vision sensor with three key advantages: a measurement rate that is almost 1 million times faster than standard cameras, a latency of 1 microsecond, and a high dynamic range of 130 decibels (standard cameras only have 60 dB). These properties enable the design of a new class of algorithms for high-speed robotics, where standard cameras suffer from motion blur and high latency. All the data are released both as text files and binary (i.e., rosbag) files.

Links:
More on event-based vision research at our lab
Tutorial on event-based vision

When using the data in an academic context, please cite the following paper.

We provide all datasets in two formats: text files and binary files (rosbag). While their content is identical, some of them are better suited for particular applications. For prototyping, inspection, and testing we recommend to use the text files, since they can be loaded easily using Python or Matlab. The binary rosbag files are intended for users familiar with the Robot Operating System (ROS) and for applications that are intended to be executed on a real system. The format is the one used by the RPG DVS ROS driver. The details about both format follows below.

Text Files

The format of the text files is as follows.

• events.txt: One event per line (timestamp x y polarity)
• images.txt: One image reference per line (timestamp filename)
• images/00000001.png: Images referenced from images.txt
• imu.txt: One measurement per line (timestamp ax ay az gx gy gz)
• groundtruth.txt: One ground truth measurements per line (timestamp px py pz qx qy qz qw)
• calib.txt: Camera parameters (fx fy cx cy k1 k2 p1 p2 k3)

Binary Files (rosbag)

The rosbag files contain the events using dvs_msgs/EventArray message types. The images, camera calibration, and IMU measurements use the standard sensor_msgs/Image, sensor_msgs/CameraInfo, and sensor_msgs/Imu message types, respectively. Ground truth is provided as geometry_msgs/PoseStamped message type.

Plots

We provide various plots for each dataset for a quick inspection. The plots are available inside a ZIP file and contain, if available, the following quantities:

• the event rate (in events/s). In spite of the sensor being asynchronous, and therefore does not have a well-defined event rate, we provide a measurement of such a quantity by computing the rate of events using intervals of fixed duration (1 ms),
• the IMU linear acceleration (in m/s²) along each axis, in the camera frame,
• the IMU gyroscopic measurement (angular velocity, in degrees/s) in the camera frame,
• the ground truth pose of the camera (position and orientation), in the frame of the motion-capture system,
• the ground truth pose of the camera (position and orientation), with respect to the first camera pose, i.e., in the camera frame.

The orientation is provided using Euler angles, according to the ZYX convention (default of MATLAB).

These datasets were generated using a DAVIS240C from iniLabs. They contain the events, images, IMU measurements, and camera calibration from the DAVIS as well as ground truth from a motion-capture system. The datasets using a motorized linear slider neither contain motion-capture information nor IMU measurements, however ground truth is provided by the linear slider's position. All datasets in gray use the same intrinsic calibration and the "calibration" dataset provides the option to use other camera models.

We provide two synthetic scenes. These datasets were generated using the event-camera simulator described below.

In addition to the datasets, we also release a simulator based on Blender to generate synthetic datasets. The simulator is useful to prototype visual-odometry or event-based feature tracking algorithms. It is described in more detail in the accompanying paper. The contrast threshold is configurable. However, it does currently not feature a model of the sensor noise. The simulator can be found on GitHub and includes a ready-to-run example.

These references point to papers that used these datasets in their experiments. If you used these data, please send your paper to mueggler (at) ifi (dot) uzh (dot) ch.

This datasets are released under the Creative Commons license (CC BY-NC-SA 3.0), which is free for non-commercial use (including research).

This work was supported by the DARPA FLA Program, the Google Faculty Research Award, the Qualcomm Innovation Fellowship, the National Centre of Competence in Research Robotics (NCCR), the Swiss National Science Foundation, and the UZH Forschungskredit.