TY - GEN
T1 - The wonderful wizard of LoC
T2 - 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2020, co-located with SPLASH 2020
AU - Alpernas, Kalev
AU - Feldman, Yotam M.Y.
AU - Peleg, Hila
N1 - Publisher Copyright: © 2020 Owner/Author.
PY - 2020/11/8
Y1 - 2020/11/8
N2 - Lines-of-code metrics (loc) are commonly reported in Programming Languages (PL), Software Engineering (SE), and Systems papers. This convention has several different, often contradictory, goals, including demonstrating the 'hardness' of a problem, and demonstrating the 'easiness' of a problem. In many cases, the reporting of loc metrics is done not with a clearly communicated intention, but instead in an automatic, checkbox-ticking, manner. In this paper we investigate the uses of code metrics in PL, SE, and System papers. We consider the different goals that reporting metrics aims to achieve, several various domains wherein metrics are relevant, and various alternative metrics and their pros and cons for the different goals and domains. We argue that communicating claims about research software is usually best achieved not by reporting quantitative metrics, but by reporting the qualitative experience of researchers, and propose guidelines for the cases when quantitative metrics are appropriate. We end with a case study of the one area in which lines of code are not the default measurement-code produced by papers' solutions-and identify how measurements offered are used to support an explicit claim about the algorithm. Inspired by this positive example, we call for other cogent measures to be developed to support other claims authors wish to make.
AB - Lines-of-code metrics (loc) are commonly reported in Programming Languages (PL), Software Engineering (SE), and Systems papers. This convention has several different, often contradictory, goals, including demonstrating the 'hardness' of a problem, and demonstrating the 'easiness' of a problem. In many cases, the reporting of loc metrics is done not with a clearly communicated intention, but instead in an automatic, checkbox-ticking, manner. In this paper we investigate the uses of code metrics in PL, SE, and System papers. We consider the different goals that reporting metrics aims to achieve, several various domains wherein metrics are relevant, and various alternative metrics and their pros and cons for the different goals and domains. We argue that communicating claims about research software is usually best achieved not by reporting quantitative metrics, but by reporting the qualitative experience of researchers, and propose guidelines for the cases when quantitative metrics are appropriate. We end with a case study of the one area in which lines of code are not the default measurement-code produced by papers' solutions-and identify how measurements offered are used to support an explicit claim about the algorithm. Inspired by this positive example, we call for other cogent measures to be developed to support other claims authors wish to make.
KW - lines of code
KW - loc
KW - research papers
UR - http://www.scopus.com/inward/record.url?scp=85097709207&partnerID=8YFLogxK
U2 - https://doi.org/10.1145/3426428.3426921
DO - https://doi.org/10.1145/3426428.3426921
M3 - منشور من مؤتمر
T3 - Onward! 2020 - Proceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Co-located with SPLASH 2020
SP - 146
EP - 156
BT - Onward! 2020 - Proceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Co-located with SPLASH 2020
A2 - Kell, Stephen
A2 - Verna, Didier
Y2 - 18 November 2020 through 20 November 2020
ER -