Introduction to Benchmarking (II): The current situation in other imaging communities

CHALLENGE

PUBLIC DATASET

BIOIMAGING

PATTERN RECOGNITION & DATA MINING& COMPUTATIONAL LEARNING

PUBLIC DATASET

In the nut shell

It aims to establish the state-of-the-art in microarray data analysis, as well as identify progress and highlight promising directions for future efforts.

CAMDA

Every year, thousands of papers are published that describe new algorithms to be applied to medical images. But very few provide a fair and direct comparison of the newly proposed algorithm with the state of the art. We believe that such comparisons can help

to substantially improve the performance of medical image analysis algorithms.

Grand Challenge in Medical Imaging Analysis

Its purposes are:To encourage research on algorithms that scale to commercial sizesTo provide a reference dataset for evaluating research […]

Million Song Dataset

To exploit the competitive nature of many researchers and drive progress forwards at a faster pace.

PASCAL

KEY POINTS FOR A CHALLENGE

• To engage the community• To generate or collect some datasets• To identify the tasks under evaluation• To define the figures of merit• To plan a schedule