TY - JOUR
T1 - Automated Analysis of Stereotypical Movements in Videos of Children With Autism Spectrum Disorder
AU - Barami, Tal
AU - Manelis-Baram, Liora
AU - Kaiser, Hadas
AU - Ilan, Michal
AU - Slobodkin, Aviv
AU - Hadashi, Ofri
AU - Hadad, Dor
AU - Waissengreen, Danel
AU - Nitzan, Tanya
AU - Menashe, Idan
AU - Michaelovsky, Analya
AU - Begin, Michal
AU - Zachor, Ditza A.
AU - Sadaka, Yair
AU - Koler, Judah
AU - Zagdon, Dikla
AU - Meiri, Gal
AU - Azencot, Omri
AU - Sharf, Andrei
AU - Dinstein, Ilan
N1 - Publisher Copyright: © 2024 Barami T et al.
PY - 2024/9/3
Y1 - 2024/9/3
N2 - Importance: Stereotypical motor movements (SMMs) are a form of restricted and repetitive behavior, which is a core symptom of autism spectrum disorder (ASD). Current quantification of SMM severity is extremely limited, with studies relying on coarse and subjective caregiver reports or laborious manual annotation of short video recordings. Objective: To assess the utility of a new open-source AI algorithm that can analyze extensive video recordings of children and automatically identify segments with heterogeneous SMMs, thereby enabling their direct and objective quantification. Design, Setting, and Participants: This retrospective cohort study included 241 children (aged 1.4 to 8.0 years) with ASD. Video recordings of 319 behavioral assessments carried out at the Azrieli National Centre for Autism and Neurodevelopment Research in Israel between 2017 and 2021 were extracted. Behavioral assessments included cognitive, language, and autism diagnostic observation schedule, 2nd edition (ADOS-2) assessments. Data were analyzed from October 2020 to May 2024. Exposures: Each assessment was recorded with 2 to 4 cameras, yielding 580 hours of video footage. Within these extensive video recordings, manual annotators identified 7352 video segments containing heterogeneous SMMs performed by different children (21.14 hours of video). Main outcomes and measures: A pose estimation algorithm was used to extract skeletal representations of all individuals in each video frame and was trained an object detection algorithm to identify the child in each video. The skeletal representation of the child was then used to train an SMM recognition algorithm using a 3 dimensional convolutional neural network. Data from 220 children were used for training and data from the remaining 21 children were used for testing. Results: Among 319 behavioral assessment recordings from 241 children (172 [78%] male; mean [SD] age, 3.97 [1.30] years), the algorithm accurately detected 92.53% (95% CI, 81.09%-95.10%) of manually annotated SMMs in our test data with 66.82% (95% CI, 55.28%-72.05%) precision. Overall number and duration of algorithm-identified SMMs per child were highly correlated with manually annotated number and duration of SMMs (r = 0.8; 95% CI, 0.67-0.93; P < .001; and r = 0.88; 95% CI, 0.74-0.96; P < .001, respectively). Conclusions and relevance: This study suggests the ability of an algorithm to identify a highly diverse range of SMMs and quantify them with high accuracy, enabling objective and direct estimation of SMM severity in individual children with ASD.
AB - Importance: Stereotypical motor movements (SMMs) are a form of restricted and repetitive behavior, which is a core symptom of autism spectrum disorder (ASD). Current quantification of SMM severity is extremely limited, with studies relying on coarse and subjective caregiver reports or laborious manual annotation of short video recordings. Objective: To assess the utility of a new open-source AI algorithm that can analyze extensive video recordings of children and automatically identify segments with heterogeneous SMMs, thereby enabling their direct and objective quantification. Design, Setting, and Participants: This retrospective cohort study included 241 children (aged 1.4 to 8.0 years) with ASD. Video recordings of 319 behavioral assessments carried out at the Azrieli National Centre for Autism and Neurodevelopment Research in Israel between 2017 and 2021 were extracted. Behavioral assessments included cognitive, language, and autism diagnostic observation schedule, 2nd edition (ADOS-2) assessments. Data were analyzed from October 2020 to May 2024. Exposures: Each assessment was recorded with 2 to 4 cameras, yielding 580 hours of video footage. Within these extensive video recordings, manual annotators identified 7352 video segments containing heterogeneous SMMs performed by different children (21.14 hours of video). Main outcomes and measures: A pose estimation algorithm was used to extract skeletal representations of all individuals in each video frame and was trained an object detection algorithm to identify the child in each video. The skeletal representation of the child was then used to train an SMM recognition algorithm using a 3 dimensional convolutional neural network. Data from 220 children were used for training and data from the remaining 21 children were used for testing. Results: Among 319 behavioral assessment recordings from 241 children (172 [78%] male; mean [SD] age, 3.97 [1.30] years), the algorithm accurately detected 92.53% (95% CI, 81.09%-95.10%) of manually annotated SMMs in our test data with 66.82% (95% CI, 55.28%-72.05%) precision. Overall number and duration of algorithm-identified SMMs per child were highly correlated with manually annotated number and duration of SMMs (r = 0.8; 95% CI, 0.67-0.93; P < .001; and r = 0.88; 95% CI, 0.74-0.96; P < .001, respectively). Conclusions and relevance: This study suggests the ability of an algorithm to identify a highly diverse range of SMMs and quantify them with high accuracy, enabling objective and direct estimation of SMM severity in individual children with ASD.
UR - http://www.scopus.com/inward/record.url?scp=85204071315&partnerID=8YFLogxK
U2 - https://doi.org/10.1001/jamanetworkopen.2024.32851
DO - https://doi.org/10.1001/jamanetworkopen.2024.32851
M3 - Article
C2 - 39264628
SN - 2574-3805
VL - 7
SP - e2432851
JO - JAMA network open
JF - JAMA network open
IS - 9
ER -