You are here

RoboCupRescue Interleague Mapping Challenge

A challenge to evaluate the performance of mapping algorithms
independent of auxiliary factors, using recorded sensor
data in a consistent computation environment (virtual machine).

Overview

Teams are asked to provide their mapping solution to the judges. Quite often those programs rely on a multitude of external libraries and are compiled for different operating systems. Therefore teams are asked to provide their software together with their OS as a virtual machine image. The judges will then start the provided systems on their computer. There they will upload the dataset, start the mapping software and gather the generated map. Those maps will then be scored judged.

There is no need to provide the source code for any software - closed source solutions are welcome to participate.

There is also no constraint regarding the programming language or environment used. All solutions are also welcome as long as they can be started from command line. It is also ok for programs to open a GUI etc. - as long as they automatically close it when the mapping task is done.

Teams that already have the capability to run their mapping algorithm on the command line from recorded data should have no problem to implement the suggested solution - it should take one person not more than one day to setup the virtual machine, install the software and use the logreader classes provided by Johannes Pellenz, Frank Neuhaus, Denis Dillenberger, Dagmar Lang, Dietrich Paulus from University of Koblenz-Landau.

The sensor data will be provided to the mapping algorithm in a defined, binary format and include data from one IMU and one LRF.

The rules document elaborates further on the setup of the image for the virtual machine, on the format of the data sets and on the format of the resulting maps (geoTIFF).

This competition is organized by the RoboCupRescue League. But it is open to all teams interested. Especially those using mapping (@Home, VirtualRescue) are encouraged to participate. Also groups currently not involved in RoboCup are very welcome.

Please contact Sören Schwertfeger (s.schwertfeger@jacobs-university.de) for comments and questions.

Results of the RoboCup Singapore 2010 Challenge

Two teams participated in the this first mapping challenge. Team CASualty from University of New South Wales, Australia, participated in the RoboCupRescue league and consists of Adam Milstein, Matthew McGill, Claude Sammut, Timothy Wiley, Rudino Saleh, Reza Farid, Mohammad Norouzi and Raymond Sheh. Download their mapping and team description here.

Team Homer @UniKoblenz together with Johannes Pellenz, Frank Neuhaus, Denis Dillenberger, Wladimir Krebs, Dagmar Lang and Dietrich Paulus from the University of Koblenz-Landau, Germany, participated in the RoboCup @Home league. Link to their mapping and team description.

Map from mappingfinal-backwards.datalog generated by CASulty.Map from mappingfinal-backwards.datalog generated by Homer@UniKoblenz.

Two example maps from CASualty (left) and Homer @UniKoblenz (right),
generated using the mappingfinal_backwards.datalog.

Map from semifinal_2.datalog generated by CASulty.Map from semifinal_2.datalog generated by Homer@UniKoblenz.

Two other example maps from CASualty (left) and Homer @UniKoblenz (right),
generated using the semifinal_2.datalog.

13 different datalogs were used to create maps. Those can be obtained here (290 MiB). The resulting maps of both teams are also available. There is also a small spreadsheet comparing and scoring the results in a simple fashion.

The winner of this competition is Team Homer@UniKoblenz - congratulations!

Future

This first competition proved that this approach for comparing mapping algorithms is working. During the RoboCup Singapore 2010 competitions and also afterwards a number of teams expressed interest to participate. Due to the short time from first announcing the interleague challenge (two weeks before the RoboCup) most teams had to concentrate on the actual competitions. Nevertheless two teams managed to change their software accordingly and install it in the virtual machine! Great job and thank you very much for the effort!

Downloads