The Cell Tracking Challenge: 10 years of objective benchmarking

Martin Maška; Vladimír Ulman; Pablo Delgado-Rodriguez; Estibaliz Gómez-de-Mariscal; Tereza Nečasová; Fidel A. Guerrero Peña; Tsang Ing Ren; Elliot M. Meyerowitz; Tim Scherr; Katharina Löffler; Ralf Mikut; Tianqi Guo; Yin Wang; Jan P. Allebach; Rina Bao; Noor M. Al-Shakarji; Gani Rahmon; Imad Eddine Toubal; Kannappan Palaniappan; Filip Lux; Petr Matula; Ko Sugawara; Klas E.G. Magnusson; Layton Aho; Andrew R. Cohen; Assaf Arbelle; Tal Ben-Haim; Tammy Riklin Raviv; Fabian Isensee; Paul F. Jäger; Klaus H. Maier-Hein; Yanming Zhu; Cristina Ederra; Ainhoa Urbiola; Erik Meijering; Alexandre Cunha; Arrate Muñoz-Barrutia; Michal Kozubek; Carlos Ortiz-de-Solórzano

doi:https://doi.org/10.1038/s41592-023-01879-y

The Cell Tracking Challenge: 10 years of objective benchmarking

Martin Maška, Vladimír Ulman, Pablo Delgado-Rodriguez, Estibaliz Gómez-de-Mariscal, Tereza Nečasová, Fidel A. Guerrero Peña, Tsang Ing Ren, Elliot M. Meyerowitz, Tim Scherr, Katharina Löffler, Ralf Mikut, Tianqi Guo, Yin Wang, Jan P. Allebach, Rina Bao, Noor M. Al-Shakarji, Gani Rahmon, Imad Eddine Toubal, Kannappan Palaniappan, Filip LuxPetr Matula, Ko Sugawara, Klas E.G. Magnusson, Layton Aho, Andrew R. Cohen, Assaf Arbelle, Tal Ben-Haim, Tammy Riklin Raviv, Fabian Isensee, Paul F. Jäger, Klaus H. Maier-Hein, Yanming Zhu, Cristina Ederra, Ainhoa Urbiola, Erik Meijering, Alexandre Cunha, Arrate Muñoz-Barrutia, Michal Kozubek, Carlos Ortiz-de-Solórzano

Department of Electrical & Computer Engineering

Ben-Gurion University of the Negev

Research output: Contribution to journal › Article › peer-review

Abstract

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Original language	American English
Pages (from-to)	1010-1020
Number of pages	11
Journal	Nature Methods
Volume	20
Issue number	7
DOIs	https://doi.org/10.1038/s41592-023-01879-y
State	Published - 1 Jul 2023

All Science Journal Classification (ASJC) codes

Molecular Biology
Biochemistry
Biotechnology
Cell Biology

Access to Document

https://doi.org/10.1038/s41592-023-01879-y

Cite this

Maška, M., Ulman, V., Delgado-Rodriguez, P., Gómez-de-Mariscal, E., Nečasová, T., Guerrero Peña, F. A., Ren, T. I., Meyerowitz, E. M., Scherr, T., Löffler, K., Mikut, R., Guo, T., Wang, Y., Allebach, J. P., Bao, R., Al-Shakarji, N. M., Rahmon, G., Toubal, I. E., Palaniappan, K., ... Ortiz-de-Solórzano, C. (2023). The Cell Tracking Challenge: 10 years of objective benchmarking. Nature Methods, 20(7), 1010-1020. https://doi.org/10.1038/s41592-023-01879-y

@article{739e4fd9ef9b4aec9ac48a91c11d7d8c,

title = "The Cell Tracking Challenge: 10 years of objective benchmarking",

abstract = "The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.",

author = "Martin Ma{\v s}ka and Vladim{\'i}r Ulman and Pablo Delgado-Rodriguez and Estibaliz G{\'o}mez-de-Mariscal and Tereza Ne{\v c}asov{\'a} and {Guerrero Pe{\~n}a}, {Fidel A.} and Ren, {Tsang Ing} and Meyerowitz, {Elliot M.} and Tim Scherr and Katharina L{\"o}ffler and Ralf Mikut and Tianqi Guo and Yin Wang and Allebach, {Jan P.} and Rina Bao and Al-Shakarji, {Noor M.} and Gani Rahmon and Toubal, {Imad Eddine} and Kannappan Palaniappan and Filip Lux and Petr Matula and Ko Sugawara and Magnusson, {Klas E.G.} and Layton Aho and Cohen, {Andrew R.} and Assaf Arbelle and Tal Ben-Haim and Raviv, {Tammy Riklin} and Fabian Isensee and J{\"a}ger, {Paul F.} and Maier-Hein, {Klaus H.} and Yanming Zhu and Cristina Ederra and Ainhoa Urbiola and Erik Meijering and Alexandre Cunha and Arrate Mu{\~n}oz-Barrutia and Michal Kozubek and Carlos Ortiz-de-Sol{\'o}rzano",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = jul,

day = "1",

doi = "https://doi.org/10.1038/s41592-023-01879-y",

language = "American English",

volume = "20",

pages = "1010--1020",

journal = "Nature Methods",

issn = "1548-7091",

publisher = "Nature Publishing Group",

number = "7",

}

Maška, M, Ulman, V, Delgado-Rodriguez, P, Gómez-de-Mariscal, E, Nečasová, T, Guerrero Peña, FA, Ren, TI, Meyerowitz, EM, Scherr, T, Löffler, K, Mikut, R, Guo, T, Wang, Y, Allebach, JP, Bao, R, Al-Shakarji, NM, Rahmon, G, Toubal, IE, Palaniappan, K, Lux, F, Matula, P, Sugawara, K, Magnusson, KEG, Aho, L, Cohen, AR, Arbelle, A, Ben-Haim, T, Raviv, TR, Isensee, F, Jäger, PF, Maier-Hein, KH, Zhu, Y, Ederra, C, Urbiola, A, Meijering, E, Cunha, A, Muñoz-Barrutia, A, Kozubek, M & Ortiz-de-Solórzano, C 2023, 'The Cell Tracking Challenge: 10 years of objective benchmarking', Nature Methods, vol. 20, no. 7, pp. 1010-1020. https://doi.org/10.1038/s41592-023-01879-y

TY - JOUR

T1 - The Cell Tracking Challenge

T2 - 10 years of objective benchmarking

AU - Maška, Martin

AU - Ulman, Vladimír

AU - Delgado-Rodriguez, Pablo

AU - Gómez-de-Mariscal, Estibaliz

AU - Nečasová, Tereza

AU - Guerrero Peña, Fidel A.

AU - Ren, Tsang Ing

AU - Meyerowitz, Elliot M.

AU - Scherr, Tim

AU - Löffler, Katharina

AU - Mikut, Ralf

AU - Guo, Tianqi

AU - Wang, Yin

AU - Allebach, Jan P.

AU - Bao, Rina

AU - Al-Shakarji, Noor M.

AU - Rahmon, Gani

AU - Toubal, Imad Eddine

AU - Palaniappan, Kannappan

AU - Lux, Filip

AU - Matula, Petr

AU - Sugawara, Ko

AU - Magnusson, Klas E.G.

AU - Aho, Layton

AU - Cohen, Andrew R.

AU - Arbelle, Assaf

AU - Ben-Haim, Tal

AU - Raviv, Tammy Riklin

AU - Isensee, Fabian

AU - Jäger, Paul F.

AU - Maier-Hein, Klaus H.

AU - Zhu, Yanming

AU - Ederra, Cristina

AU - Urbiola, Ainhoa

AU - Meijering, Erik

AU - Cunha, Alexandre

AU - Muñoz-Barrutia, Arrate

AU - Kozubek, Michal

AU - Ortiz-de-Solórzano, Carlos

PY - 2023/7/1

Y1 - 2023/7/1

N2 - The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

AB - The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85159660888&partnerID=8YFLogxK

U2 - https://doi.org/10.1038/s41592-023-01879-y

DO - https://doi.org/10.1038/s41592-023-01879-y

M3 - Article

C2 - 37202537

SN - 1548-7091

VL - 20

SP - 1010

EP - 1020

JO - Nature Methods

JF - Nature Methods

IS - 7

ER -

The Cell Tracking Challenge: 10 years of objective benchmarking

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this