Login | Register

CREMI

MICCAI Challenge on
Circuit Reconstruction from Electron Microscopy Images

News

Tuesday, April 18, 2017:
We release the trained network, affinities, and re-aligned training and testing volumes of submission MALA v2. The archive is available here (33GB).
Monday, October 17, 2016:
#CREMIChallenge satellite event of MICCAI 2016 starts at 2pm in Omikron II.
Monday, September 12, 2016:
The schedule for our on-site event is online.
Friday, September 8, 2016:
The leaderboard is online.
Wednesday, September 7, 2016:
We are currently evaluating the submissions. The results will be released here soon, and all participants will be notified by mail.
Thursday, June 16, 2016:
#CREMIChallenge evaluation metrics description and code released.
Wednesday, June 1, 2016:
The test data is released and the submission system is online.
Thursday, May 26, 2016:
Janelia is doing infrastructure maintenance during the Memorial day weekend. We expect cremi.org to go up and down from Friday, May 27, 7PM through Tuesday, May 31, 6AM.
Wednesday, May 4, 2016:
#CREMIChallenge python scripts to read/write CREMI datasets are on GitHub.
Sunday, May 1, 2016:
First version of training data released.

Schedule

The on-site event at MICCAI 2016 will take place Monday, October 17th, 14:00-18:00, in Omikron II, with the following program:

14:00-14:30
Introduction, Jan Funke
14:30-15:00
Keynote I, Juan Nunez-Iglesias (VLSCI, University of Melbourne, Australia)
15:00-15:30
Presentation by group IAL, category neuron segmentation
15:30-16:00
Keynote II, Anna Kreshuk (HCI Heidelberg)
16:00-16:30
Coffee break
16:30-17:00
Presentation by group DIVE, category neuron segmentation and synaptic cleft detection
17:00-17:30
Presentation by group SDG, category synaptic cleft detection
17:30-18:00
Keynote III, Srinivas Turaga (Janelia Research Campus)

About

The goal of this challenge is to evaluate algorithms for automatic reconstruction of neurons and neuronal connectivity from serial section electron microscopy data. The comparison is performed not only by evaluating the quality of neuron segmentations, but also by assessing the accuracy of detecting synapses and identifying synaptic partners. The challenge is carried out on three large and diverse datasets from adult Drosophila melanogaster brain tissue, comprising neuron segmentation ground truth and annotations for synaptic connections. A successful solution would demonstrate its efficiency and generalizability, and carry great potential to reduce the time spent on manual reconstruction of neural circuits in electron microscopy volumes.

Datasets and Challenge Categories

We provide three datasets, each consisting of two (5 μm)3 volumes (training and testing, each 1250 px × 1250 px × 125 px) of serial section EM of the adult fly brain. Each volume has neuron and synapse labelings and annotations for pre- and post-synaptic partners.

Submissions can be made in the following categories:

Segmentation
Assessment of neuron labelings in terms of pixel-based measures (Rand index and variation of information) and topological correctness (Tolerant Edit Distance). Winners will be determined using a weighted average of the scores.
Synapse Detection
Detection of synapses. Winners are determined using the F-measure of false positive and false negative detections.
Connectivity
Identification of synaptic partners. Winners are determined using the F-measure of false positive and false negative detections of pre- and post-synaptic pairs.

Details about the evaluation procedures will be released soon.

Important Dates

May 1
Opening of training phase    ✔
June 1
Opening of testing phase    ✔
August 31
Submission deadline for test results (11:59 PM PDT)    ✔
September 7
Announcement of results and submission deadline for extended abstracts    ✔
October 17
Oral presentations of participants at MICCAI 2016 in Athens

After the on-site event at MICCAI 2016, the results of the challenge will be summarized in a review paper. The challenge will stay open for submission.

Organizers

Please contact the organizers at challenge@cremi.org.