Abstract
The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.
| Original language | English |
|---|---|
| Pages (from-to) | 1010-1020 |
| Number of pages | 11 |
| Journal | Nature Methods |
| Volume | 20 |
| Issue number | 7 |
| DOIs | |
| State | Published - 1 Jul 2023 |
All Science Journal Classification (ASJC) codes
- Molecular Biology
- Biochemistry
- Biotechnology
- Cell Biology
Access to Document
Other files and links
Fingerprint
Dive into the research topics of 'The Cell Tracking Challenge: 10 years of objective benchmarking'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver
}
In: Nature Methods, Vol. 20, No. 7, 01.07.2023, p. 1010-1020.
Research output: Contribution to journal › Article › peer-review
TY - JOUR
T1 - The Cell Tracking Challenge
T2 - 10 years of objective benchmarking
AU - Maška, Martin
AU - Ulman, Vladimír
AU - Delgado-Rodriguez, Pablo
AU - Gómez-de-Mariscal, Estibaliz
AU - Nečasová, Tereza
AU - Guerrero Peña, Fidel A.
AU - Ren, Tsang Ing
AU - Meyerowitz, Elliot M.
AU - Scherr, Tim
AU - Löffler, Katharina
AU - Mikut, Ralf
AU - Guo, Tianqi
AU - Wang, Yin
AU - Allebach, Jan P.
AU - Bao, Rina
AU - Al-Shakarji, Noor M.
AU - Rahmon, Gani
AU - Toubal, Imad Eddine
AU - Palaniappan, Kannappan
AU - Lux, Filip
AU - Matula, Petr
AU - Sugawara, Ko
AU - Magnusson, Klas E.G.
AU - Aho, Layton
AU - Cohen, Andrew R.
AU - Arbelle, Assaf
AU - Ben-Haim, Tal
AU - Raviv, Tammy Riklin
AU - Isensee, Fabian
AU - Jäger, Paul F.
AU - Maier-Hein, Klaus H.
AU - Zhu, Yanming
AU - Ederra, Cristina
AU - Urbiola, Ainhoa
AU - Meijering, Erik
AU - Cunha, Alexandre
AU - Muñoz-Barrutia, Arrate
AU - Kozubek, Michal
AU - Ortiz-de-Solórzano, Carlos
N1 - Funding Information: The authors thank J. Padilla Pérez, who worked for many hours on the annotation of the new datasets; J.-Y. Tinevez, who kindly added the CTC measures into the popular TrackMate software and who, with T. Pietzsch, developed the Mastodon software that became instrumental for us when preparing the tracking annotations of the large embryonic datasets; and the participants of the challenge not included in the list of authors of this analysis paper, listed on the challenge website (http://celltrackingchallenge.net/participants/). This work was funded by the Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación (MCIU/AEI/10.13039/50110011033) and FEDER funds UE under Grants RTI2018-094494-B-C22, TED2021-131300B-I00, PDI2021-122409OB-C22 (C.O.S.); Czech Ministry of Education, Youth and Sports national research infrastructure Czech-BioImaging projects LM2023050 and CZ.02.1.01/0.0/0.0/18_046/0016045 (M.M., V.U. and M.K.); Czech Science Foundation (GACR) grant GA21-20374S (P.M. and F.L.); European Regional Development Fund in the IT4Innovations national super-computing center–path to exascale project CZ.02.1.01/0.0/0.0/16_013/0001791 within the Operational Programme Research, Development and Education (V.U.); Czech Ministry of Education, Youth and Sports through the e-INFRA CZ project ID:90140 (V.U.); Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación Grant PID2019-109820RB-I00, MCIN/AEI/10.13039/501100011033/, co-financed by the European Regional Development Fund (ERDF), ‘A way of making Europe’ (P.D.-R., E.G.M., grant to A.M.-B.); BBVA Foundation under a 2017 Leonardo Grant for Researchers and Cultural Creators (A.M.-B.). NVIDIA Corporation for the donation of the Titan X (Pascal) GPU (P.D.-R., E.G.M., to A.M.-B.); the Gulbenkian Foundation, the European Molecular Biology Organization (EMBO) Installation Grant (EMBO-2020-IG-4734) (granted to the Optical Cell Biology laboratory at Instituto Gulbenkian de Ciência) and Postdoctoral Fellowship (EMBO ALTF 174-2022) (E.G.M.); Helmholtz Association program NACIP – Natural, Artificial and Cognitive Information Processing and Biointerfaces International Graduate School (BIF-IGS) (T.S. and R.M.), and HIDSS4Health – Helmholtz Information and Data Science School for Health (K.L. and R.M.); European Research Council, under the European Union Horizon 2020 programme, grant ERC-2015-AdG 694918 (K.S.); Helmholtz Imaging (F.I., P.F.J.); the Negev scholarship at Ben-Gurion University (A.A.); the Kreitman School of Advanced Graduate Studies (A.A.); Israel Ministry of Science, Technology and Space (MOST 3-14344 T.R.R.); the United States – Israel Binational Science Foundation (BSF 2019135 T.R.R.); Human Frontiers Science Program Grant RGP0043/2019 (A.R.C. and L.A.); the Brazilian funding agencies FACEPE, CAPES and CNPq (F.A.G.P., T.I.R.); Beckman Institute at Caltech (A.C., F.A.G.P.); Howard Hughes Medical Institute (E.M.M.); and the USA NIH NINDS R01NS110915 (K.P.) and USA ARL W911NF-18-20285 (K.P.). Funding Information: The authors thank J. Padilla Pérez, who worked for many hours on the annotation of the new datasets; J.-Y. Tinevez, who kindly added the CTC measures into the popular TrackMate software and who, with T. Pietzsch, developed the Mastodon software that became instrumental for us when preparing the tracking annotations of the large embryonic datasets; and the participants of the challenge not included in the list of authors of this analysis paper, listed on the challenge website ( http://celltrackingchallenge.net/participants/ ). This work was funded by the Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación (MCIU/AEI/10.13039/50110011033) and FEDER funds UE under Grants RTI2018-094494-B-C22, TED2021-131300B-I00, PDI2021-122409OB-C22 (C.O.S.); Czech Ministry of Education, Youth and Sports national research infrastructure Czech-BioImaging projects LM2023050 and CZ.02.1.01/0.0/0.0/18_046/0016045 (M.M., V.U. and M.K.); Czech Science Foundation (GACR) grant GA21-20374S (P.M. and F.L.); European Regional Development Fund in the IT4Innovations national super-computing center–path to exascale project CZ.02.1.01/0.0/0.0/16_013/0001791 within the Operational Programme Research, Development and Education (V.U.); Czech Ministry of Education, Youth and Sports through the e-INFRA CZ project ID:90140 (V.U.); Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación Grant PID2019-109820RB-I00, MCIN/AEI/10.13039/501100011033/, co-financed by the European Regional Development Fund (ERDF), ‘A way of making Europe’ (P.D.-R., E.G.M., grant to A.M.-B.); BBVA Foundation under a 2017 Leonardo Grant for Researchers and Cultural Creators (A.M.-B.). NVIDIA Corporation for the donation of the Titan X (Pascal) GPU (P.D.-R., E.G.M., to A.M.-B.); the Gulbenkian Foundation, the European Molecular Biology Organization (EMBO) Installation Grant (EMBO-2020-IG-4734) (granted to the Optical Cell Biology laboratory at Instituto Gulbenkian de Ciência) and Postdoctoral Fellowship (EMBO ALTF 174-2022) (E.G.M.); Helmholtz Association program NACIP – Natural, Artificial and Cognitive Information Processing and Biointerfaces International Graduate School (BIF-IGS) (T.S. and R.M.), and HIDSS4Health – Helmholtz Information and Data Science School for Health (K.L. and R.M.); European Research Council, under the European Union Horizon 2020 programme, grant ERC-2015-AdG 694918 (K.S.); Helmholtz Imaging (F.I., P.F.J.); the Negev scholarship at Ben-Gurion University (A.A.); the Kreitman School of Advanced Graduate Studies (A.A.); Israel Ministry of Science, Technology and Space (MOST 3-14344 T.R.R.); the United States – Israel Binational Science Foundation (BSF 2019135 T.R.R.); Human Frontiers Science Program Grant RGP0043/2019 (A.R.C. and L.A.); the Brazilian funding agencies FACEPE, CAPES and CNPq (F.A.G.P., T.I.R.); Beckman Institute at Caltech (A.C., F.A.G.P.); Howard Hughes Medical Institute (E.M.M.); and the USA NIH NINDS R01NS110915 (K.P.) and USA ARL W911NF-18-20285 (K.P.). Publisher Copyright: © 2023, The Author(s).
PY - 2023/7/1
Y1 - 2023/7/1
N2 - The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.
AB - The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.
UR - http://www.scopus.com/inward/record.url?scp=85159660888&partnerID=8YFLogxK
U2 - https://doi.org/10.1038/s41592-023-01879-y
DO - https://doi.org/10.1038/s41592-023-01879-y
M3 - Article
C2 - 37202537
SN - 1548-7091
VL - 20
SP - 1010
EP - 1020
JO - Nature Methods
JF - Nature Methods
IS - 7
ER -