Why are only some children with autism spectrum disorder misclassified by the social communication questionnaire? An empirical investigation of individual differences in sensitivity and specificity in a clinic-referred sample
Journal of Neurodevelopmental Disorders volume 15, Article number: 28 (2023)
The Social Communication Questionnaire (SCQ) is a checklist for autism spectrum disorder (ASD) commonly used in research and clinical practice. While the original validation study suggested that the SCQ was an accurate ASD screener with satisfactory sensitivity and specificity, subsequent studies have yielded mixed results, with some revealing low sensitivity, low specificity, and low utility in some settings.
The present study examined the psychometric properties of the SCQ as well as the individual difference characteristics of 187 individuals with and without autism spectrum disorder (ASD) who were misclassified or accurately classified by the SCQ in a clinic-referred sample.
The SCQ showed suboptimal sensitivity and specificity, regardless of age and sex. Compared to true positives, individuals classified as false positives displayed greater externalizing and internalizing problems, whereas individuals classified as false negatives displayed better social communication and adaptive skills.
The findings suggest that non-autistic developmental and behavioral individual difference characteristics may explain high rates of misclassification using the SCQ. Clinicians and researchers could consider using the SCQ in combination with other tools for young children with internalizing and externalizing symptoms and other more complex clinical presentations.
Heterogeneity in symptom presentation makes accurate screening and assessing for autism spectrum disorder (ASD) particularly challenging. While a variety of measures have been developed to identify children at risk for ASD, a commonly used tool in research and clinical practice is the Social Communication Questionnaire (SCQ, formerly named Autism Screening Questionnaire) . The SCQ was developed to be a reliable and valid screener for ASD . It has been widely used in the clinical setting for triage and referral decision-making, as well as in research settings to serve as a screening tool for inclusion or exclusion criteria. Studies examining the SCQ have yielded mixed results, with some suggesting low sensitivity, low specificity, and low utility within certain clinical settings due to its unsatisfactory psychometric properties [3,4,5,6,7,8,9]. Given the recent literature suggesting that the SCQ may not be uniformly useful as a screening tool for ASD, research is needed to better understand factors that might be affecting SCQ’s sensitivity and specificity.
In a recent meta-analysis that systematically assessed the accuracy of the SCQ over the last 15 years, the authors concluded that variability in sampling methods impacted the accuracy of the SCQ for ASD screening . Indeed, the SCQ had significantly higher sensitivity and specificity in samples that were population reference samples compared to clinical samples, community samples, and convenience samples, and the effect size was the biggest when comparing population samples to clinical samples . Moreover, a recent study of a clinic-referred sample of verbally fluent children and adolescents found that the SCQ performed no better than chance in correctly classifying children with ASD and those with non-spectrum diagnoses . The poor performance of the SCQ in accurately classifying ASD among clinical samples further suggested that individual characteristics may influence variability in the SCQ’s effectiveness as a screening tool [12, 13].
An emerging literature suggests that individual variables that may explain variability in SCQ effectiveness include the demographic factors of age and sex. Several studies reveal lower accuracy for the suggested SCQ cut-offs at younger ages. Maintaining a cut-off of 15, the SCQ has demonstrated satisfactory results in school-age populations [5, 14]; however, the sensitivity and specificity hover between 0.50 to 0.75 for younger populations [4, 5, 7, 10]. For children under 8 years old, Corsello et al.  also reported low sensitivity and specificity for the published cut-off of 15 and suggested lowering the SCQ cut-off score to ≥ 11 (for age under 5) or ≥ 12 (for age 5–7) to achieve better sensitivity. Subsequent studies confirmed a cut-off score of 11 maximizes sensitivity and specificity (a better balance between these two critical indices) in young children [13, 15,16,17]. Another study found that the recommended cut-off of 15 had accurate sensitivity and specificity for adolescents and adults (ages 13–21), but a cut-off of 11 worked best for children ages 4 to 12 years .
Sex differences have also been found on assessment measures for ASD [18, 19], although only a few studies have examined sex differences in SCQ accuracy. Evans and colleagues found that the SCQ performed adequately in boys (sensitivity 0.77; specificity 0.66) and well in girls (sensitivity 1; specificity 0.71) among 272 school-age children . However, in a multisite case-control study of young children with either ASD or non-spectrum developmental disabilities, males scored significantly higher than females on the SCQ in both groups . Several studies did not compare sensitivity and specificity directly but found similar SCQ mean total scores for males and females [2, 3, 5, 13]. Overall, extant research examining the impact of age and sex on SCQ accuracy has produced conflicting results, and work is needed to understand whether these demographic factors impact the effectiveness of SCQ as a screening tool.
In addition to demographic characteristics, individual differences in clinical presentations may also influence variability in the SCQ results [12, 13]. Previous studies have continued to find poor sensitivity–specificity balance in distinguishing between ASD and other developmental disorders among children [21, 22]. Suren and colleagues  found that sensitivity of the SCQ was higher for children with ASD who had not developed phrase speech and were significantly delayed in their cognitive development. Similarly, other studies have revealed satisfactory sensitivity (ranging from 0.88 to 1.00) but poor specificity (ranging from 0.17 to 0.43) in distinguishing between individuals with intellectual disabilities with and without ASD [24, 25]. Multiple studies have found a negative correlation between SCQ total scores and adaptive skills and intelligence quotient (IQ) [7, 9, 26, 27]. Findings suggest that parent report on the SCQ is related to individual differences in children’s cognitive ability, language ability, and adaptive functioning.
Internalizing and externalizing psychopathology are also candidate factors for explaining variation in SCQ effectiveness. Studies examining the clinical utility of the SCQ in samples of children with a variety of neurodevelopmental and mental health diagnoses generally have found lower sensitivity and specificity than in the original validation study [3, 7, 26, 27]. For example, using a heterogenous sample, Hollocks et al.  found generally adequate sensitivity when using the SCQ cut-offs ranging from 13 to 20, but specificity below 0.40 for all cut-offs (the recommended cut-off of 15 had a sensitivity of 0.84 and specificity of 0.13). Ung et al.  also reported suboptimal sensitivity (0.70) and specificity (0.67) at the recommended cut-off of 15, and a cut-off of 11 improved sensitivity to 0.82, at a cost to specificity (0.37). Moreover, studies comparing children with ASD and attention-deficit hyperactivity disorder (ADHD) have found adequate sensitivity and specificity of the SCQ [28, 29], while one study found the optimal cut-off of the SCQ was lower than the recommended cut-off of 15 for differentiating ASD from ADHD . These findings suggested that the SCQ scores are highly influenced by non-autism developmental and behavioral factors, which further raise questions as to whether sensitivity and specificity of the SCQ are adequate for screening purposes.
Taken together, individual differences in clinical presentation, such as cognitive ability, language ability, comorbidities, and the presence of challenging behaviors seem to be impacting the effectiveness of the SCQ. Other factors including sample referral source, sample heterogeneity, and parental understanding of ASD may also impact the performance of the SCQ. Studies to date have identified individual difference factors described above as broad categories of features that can influence the accuracy of SCQ, but more research is needed on the specific behavioral and developmental features that impact accuracy, especially those that cross diagnostic categories.
The current study aims to overcome the limitations of prior research by focusing on a heterogeneous group of children from a clinic-referred sample and examining whether individual difference characteristics differentiate those who are misclassified by the SCQ. The first aim of the present study adds to the literature on SCQ accuracy by estimating its sensitivity and specificity in a large clinical-referred sample. Based on prior literature, we hypothesize that the SCQ will show lower sensitivity and specificity than that presented in the original validation study. The second aim is to examine the individual characteristics of children misclassified by the SCQ. We hypothesize that the presence of more comorbid conditions, lower cognitive ability, greater autism severity, greater internalizing and externalizing symptoms, and lower adaptive functioning may explain why the sensitivity and specificity of the SCQ are poor in clinic-referred samples. The results of the study provide valuable information when considering using the SCQ in a clinical or research setting, especially for a population who are seeking their first-time ASD diagnosis or assessment.
Participants included individuals who received diagnostic neurodevelopmental evaluations at an autism specialty clinic, referred by neontologists, pediatricians, geneticists, pediatric neuropsychologists, psychiatrists, as well as other professionals who work closely with the families (i.e., speech pathologists, occupational therapists). Clinical diagnoses were made by licensed psychologists with extensive experience in the assessment and treatment of ASD and common co-occurring conditions (5–15 years of clinical experience). Licensed psychologists integrated multiple sources of information, including medical history, educational records, caregiver reports, neuropsychological assessments, and comprehensive interviews regarding autistic characteristics to make diagnostic decisions based on DSM-5 criteria. The SCQ was collected as part of the diagnostic evaluation but was not the sole factor considered when making a diagnosis. The methods were approved by the Institutional Review Board at the University of Minnesota.
From 2016 to 2019, data from 793 consecutive evaluations were entered into a de-identified clinical database. To examine the current study hypotheses, only initial evaluation data (N = 519) were included in the analysis. Data of individuals who were younger than 4 years old at their initial evaluations (N = 142) were excluded from analyses. Individuals who did not complete or only partially completed the SCQ (N = 162) were also excluded. Finally, only participants who had completed all the measures used in the present study (described below) were included. Ultimately, initial diagnostic evaluations from 187 children and adolescents were included in the present study.
Among all individuals, 133 (94 males and 39 females) were clinically diagnosed with ASD, and 54 (34 males and 20 females) were identified as non-spectrum with other neurodevelopmental challenges. Within the non-spectrum sample, 32 received a primary diagnosis of ADHD (59.3%), 10 were diagnosed with a language disorder (18.5%), 5 had behavioral disorders (9.3%; e.g., oppositional defiant disorder, conduct disorder, disruptive behavioral disorder), 27 had anxiety disorders (50.0%), 7 had mood disorders (13.0%), and 8 had other genetic and/or physical disabilities (14.8%; e.g., fragile X syndrome, Williams syndrome, or mild cerebral palsy).
Demographic characteristics were collected through a clinical intake questionnaire and clinical interviews with the primary caregivers. All participants were given the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2), and caregivers were administered the Autism Diagnostic Interview-Revised (ADI-R). Cognitive and language skills were assessed using clinically appropriate measures depending on the child's age and function level. Scores on individual sub-tests are standardized against age-specific norms and then grouped to produce separate measures of verbal and non-verbal IQ, with the former encompassing those tests most related to verbal skills and the latter being more independent of verbal skills. Full Scale IQ (FSIQ) is the composite of these verbal and non-verbal skills. All individuals also completed the following measures.
Social Communication Questionnaire, Lifetime Version (SCQ)
The SCQ  is a 40-item questionnaire that measures the symptomatology associated with ASD (e.g., certain communitive behaviors, language uses, and stereotyped behaviors) focusing on the behaviors that are rare in non-affected individuals, based on an established diagnostic interview, the Autism Diagnostic Interview-Revised (ADI-R) . The Lifetime SCQ version asks respondents to focus on characteristics of the individual at age 4 to 5 years for developmentally influenced behaviors, or at any point in their lifetime for behaviors that are atypical at any age (e.g., repetitive motor movements), while the Current SCQ version focuses on characteristics present within the previous 3 months . Total scores range from 0 to 39, with higher scores reflecting the presence of more symptoms.
Behavioral Assessment System for Children, Parent Rating Scale (BASC-PRS)
The BASC-PRS  is a parent-report questionnaire using a multi-dimensional approach to evaluate behaviors and adaptive skills for children ages 2 years, 6 months to 21 years. This sample completed the BASC-PRS, Third Edition, which generates four composite scales (Externalizing Problems, Internalizing Problems, Behavioral Symptoms Index, and Adaptive Skills) and 14 Primary Scales (Hyperactivity, Aggression, Conduct Problems, Anxiety, Depression, Somatization, Atypicality, Withdrawal, Attention Problems, Adaptability, Social Skills, Leadership, Activities of Daily Living, and Functional Communication). For behavioral scales, higher T scores indicate greater difficulties, whereas for adaptive scales, lower T scores represent greater challenges .
Vineland Adaptive Behavior Scale (VABS)
The VABS [32, 33] is a standardized clinical assessment tool that utilizes a semi-structured interview to measure adaptive behaviors and skills for individuals with developmental challenges. For this sample, the third edition of the VABS, Comprehensive Interview Form, was used to assess participants’ adaptive skills. The VABS consists of three subscales, including Communication (receptive, expressive, written), Socialization (interpersonal relationships, play and leisure, coping skills), and Daily Living skills (person, domestic, community), yielding an overall composite score of adaptive skills (Adaptive Behavior Composite). The VABS also provides an indirect measure of gross and fine motor skills, yielding a Motor skills domain [32, 33].
Autism Diagnostic Observation Schedule, Second Edition (ADOS-2)
The ADOS-2  is a standardized, semi-structured observational assessment used to assess language and communication, reciprocal social interaction, imagination/creativity, as well as stereotyped behaviors and restricted interests to inform diagnosis of ASD. The ADOS-2 is organized into five modules based on the individual’s expressive language level (and, in some cases, chronological age), ranging from preverbal to verbally fluent. The diagnostic algorithm provides separate total scores for the Social Affect (SA) and Restricted and Repetitive Behavior (RRB) domains, as well as a cut-off for the sum of the two domains to provide instrument classifications of autism, autism spectrum, or non-spectrum. The Calibrated Severity Score (CSS) is a standardized version of ADOS-2 raw total scores aimed to minimize the impact of factors such as age, language, and cognitive ability. The ADOS-2 CSS has been suggested as a measure of symptom severity independent of these developmental factors .
Data collected through the comprehensive evaluations were entered, coded, and checked for errors and logic using a standardized procedure. Subsequently, the entered data were transferred to SAS 9.4 (SAS Institute Inc., Cary, NC, USA) to perform range checking and internal consistency checking. Based on the clinical diagnosis of ASD and suggested cut-off score of the SCQ (summed score of 15 or above) , individuals were separated into four groups to test our hypotheses: true positives (TP; have a clinical diagnosis of ASD and the SCQ score is above 15; N = 82), false negatives (FN; have a clinical diagnosis of ASD but the SCQ score is below 15; N = 51), false positives (FP; does not have a clinical diagnosis of ASD but the SCQ score is above 15; N = 22), and true negatives (TN; does not have a clinical diagnosis of ASD and the SCQ score is below 15; N = 32). Demographic homogeneity of the groups was assessed using chi-square tests and post-hoc analyses (Fisher’s exact test) for discrete variables. Continuous variables, such as sum or mean score of cognitive measures, symptomatologic items, as well as emotional, behavioral, and adaptive measures, were compared between groups by using independent t tests and post-hoc analyses (Bonferroni correction). To determine the performance of the SCQ as a screener for ASD in this clinical-referred sample, the sensitivity (i.e., number of true positive/[true positive + false negative]) and specificity (i.e., number of true negative/[true negative + false positive]) were calculated at multiple cut-off points. In addition, the area under the curve (AUC; the area under the receiver operating characteristic (ROC) curve) is considered to be a metric of fit between the true positive rate (sensitivity) and the false positive rate (1—specificity). AUC values can range from 0.5 to 1.0, with estimates closer to 1 indicating greater accuracy. Hanley and McNeil  suggested that a screener is a failure when an AUC is between 0.5 and 0.6, poor between 0.6 and 0.7, fair between 0.7 and 0.8, acceptable between 0.8 and 0.9, and perfect between 0.9 and 1.0.
When applying the suggested cut-off score of ≥ 15 for a classification of ASD versus non-spectrum to all the samples, the AUC was 0.605, with a sensitivity of 0.677 and specificity of 0.593. When adjusting the cut-off score of the SCQ (e.g., cut-off ≥ 11 to ≥ 20), the AUC continued to reveal poor accuracy (range 0.581–0.634) with a better sensitivity when using a lower cut-off score (e.g., cut-off ≥ 11, sensitivity = 0.857, specificity = 0.315; Table 1) and a greater specificity when using a higher cut-off score (e.g., cut-off ≥ 20, sensitivity = 0.406, specificity = 0.759; Table 1).
When examining by sex, using the suggested cut-off score of ≥ 15, the SCQ achieved acceptable sensitivity among females with poor accuracy and specificity (N = 51; AUC = 0.686, sensitivity = 0.800, specificity = 0.571; Table 1), whereas AUC, sensitivity, and specificity remained poor in males (N = 136; AUC = 0.631, sensitivity = 0.675, specificity = 0.586). When adjusting the cut-off score of the SCQ (e.g., cut-off ≥ 11 to ≥ 20), the AUC among males (range 0.539–0.631) and females (AUC range 0.579–0.691) continued to reveal poor accuracy (Table 1).
To examine the discriminative validity of the SCQ with respect to age, the sample was separated into three age groups: age 4 to 7 years (N = 84), age 8 to 11 (N = 58), and age 12 and above (age range from 12.07 to 18.21; N = 45) based on previous literature (Corsello et al. 2007). In the youngest group, the AUC, sensitivity, and specificity were low when using the suggested cut-off score (cut-off ≥ 15; AUC = 0.587, sensitivity = 0.698, specificity = 0.476; Table 1), but the sensitivity improved when adjusting the cut-off to ≥ 11 (cut-off ≥ 11; AUC = 0.627, sensitivity = 0.873, specificity = 0.381; Table 1). For the older age groups, the 8 to 11 group had similar accuracy, sensitivity, and specificity to the whole group (cut-off ≥ 15; AUC = 0.638, sensitivity = 0.639, specificity = 0.636; Table 1), and age 12 and above yielded a fair AUC with better specificity (cut-off ≥ 15; AUC = 0.702, sensitivity = 0.676, specificity = 0.727; Table 1). All of the results were lower than reported in the original studies [1, 2].
Comparison between groups
False negatives (FN) vs. true negatives (TN)
Compared to the FN, TN had more individuals who received ADHD, anxiety disorders, and depressive disorders as their final diagnoses, and more individuals with two or more comorbid diagnoses (Table 2). Regarding autism symptomatology, FN exhibited more restricted and repetitive behaviors than TN based on clinician observation (ADOS RRB CSS; Table 3, Fig. 1A), but caregivers’ reports (BASC Withdrawal and Atypicality) did not reveal significant differences among the two groups (Table 3, Fig. 1B). The adaptive skills assessed by the caregiver-reported BASC and VABS revealed similar results among FN and TN, suggesting similar adaptive skills among those misclassified and non-spectrum individuals (Table 3, Fig. 1C and D). Moreover, FN reportedly demonstrated better adaptability (i.e., the ability to adapt readily to changes in the environment) than TN on the BASC (Table 3, Fig. 1C). No significant differences on cognitive functioning, internalizing problems, or externalizing issues between groups (Table 3).
False negatives (FN) vs. true positives (TP)
FN demonstrated similar comorbidity, cognitive functioning, and autism symptomatology with TP based on direct neuropsychological measures and clinicians’ observation on the ADOS-2 (Tables 2 and 3, Fig. 1A). Caregivers also reported similar internalizing and externalizing problems on the BASC among FN and TP (Table 3). Compared to TP, FN displayed less atypical behaviors and social withdrawal based on caregivers’ observation on the BASC (Table 3, Fig. 1B). FN also exhibited better adaptive skills then TP on the BASC and VABS (Table 3, Fig. 1C, D).
False positives (FP) vs. true positives (TP)
Compared to TP, FP had more individuals who were diagnosed with behavioral disorders, but fewer individuals received diagnosis of intellectual disability (Table 2). Consistent with their final diagnoses, TP displayed higher ADOS CSS than FP on SA, RRB, and total comparison score of ADOS (Table 3, Fig. 1A). However, caregivers of FP reported greater externalizing and internalizing problems than TP (Table 3). Specifically, caregivers of FP reported clinically significant scores on domains of hyperactivity, aggression, and conduct problems, and their scores on aggression and conduct problems were significantly higher than TP (Table 3, Fig. 1B). Caregivers of FP also reported greater internalizing problems, especially anxiety, which were significantly higher than TP group (Table 3, Fig. 1B). In general, caregivers of FP reported similar adaptive skills with TP across domains on the BASC and the VABS (Table 3, Fig. 1C, D), though FP reportedly displayed better daily living skills than TP on the VABS, resulted in greater overall Adaptive Behavioral Composite than TP (Table 3, Fig. 1D).
False positives (FP) vs. false negatives (FN)
Compared to FN, FP had more individuals who received ADHD and behavioral disorders as their final diagnoses among this sample, and more individuals with two or more diagnoses (Table 2). Consistent with their final diagnoses by the licensed psychologists, FN displayed higher ADOS CSS than FP on SA, RRB, and total comparison score of ADOS (Table 3, Fig. 1A). When examining internalizing and externalizing symptoms on the BASC, caregivers of FP reported greater behavioral issues than FN, including aggression and conduct problems (Table 3, Fig. 1B). The FP group also exhibited more atypical behaviors than FN (Table 3, Fig. 1B). In terms of adaptive skills, caregivers of FN and FP generally reported similar adaptive skills on the BASC and VABS (Table 3, Fig. 1C, D), with an exception that FN reportedly demonstrated better adaptability than FP based on the BASC (Table 3, Fig. 1C).
The present study examined the behavioral characteristics of children who were misclassified by the SCQ in a clinic-referred sample. Among the 187 participants in the present study, only 65% of individuals were accurately classified, revealing low sensitivity (0.68) and specificity (0.59). Individuals with ASD who were accurately classified on the SCQ (true positives) had lower adaptive skills and higher ADOS-2 scores than other groups, including false negatives. True positives did not differ from false negatives (or other groups) in terms of internalizing or externalizing behaviors. Individuals who were falsely classified as ASD (false positives) displayed more externalizing (e.g., aggression and conduct problems) and internalizing psychopathology (e.g., anxiety) and were more likely to be diagnosed with behavioral disorders (e.g., oppositional defiant disorder, conduct disorder) compared to the other groups. Regarding adaptive skills, while the non-spectrum group (true negatives and false positives) in general demonstrated higher adaptive skills than the ASD group (true positives and false negatives), the false negative group had a similar level of adaptive skills (e.g., communication, daily living skills, social skills, leadership, and functional communication) to the non-spectrum group.
Our results make an important contribution to knowledge about the limitations of classification properties of the SCQ in discriminating autism from other conditions in a referral population who seek their first diagnosis. Individuals who were falsely classified as ASD exhibited more externalizing (e.g., aggression and conduct problems) and internalizing issues (e.g., anxiety) compared to other groups. Moreover, they had comparable social impairments and atypical behaviors compared to children in the true positive group. It is likely that there may be a subset of children with more severe behavioral problems who also show early and continued social impairment, secondary to difficulty controlling impulses and regulating emotions. As a result, the caregivers may have endorsed numerous items on the SCQ to reflect the social challenges and general difficulties their children faced. Alternatively, another possibility is that parents are identifying atypical presentations in early development but are unable to clearly differentiate the types of difficulties they are observing on a yes/no questionnaire, thus reporting several social impairments that may better reflect current behavioral challenges. Since these behavioral challenges are not specifically differentiated and measured on the SCQ, the reporters may not have adequate places to voice their concerns, resulting in elevated scores. It is important to note that anxiety and depression in young children can be difficult for parents to differentiate from autistic behaviors, as internalizing symptoms may present as social withdrawal, irritability, rigidity, preference for routine and order, and repetitive behaviors, which are also symptoms of ASD [37, 38]. In fact, there is a known challenge of differentiating autistic symptomology from developmental and behavioral presentations, even when utilizing a standardized diagnostic measure such as the ADOS by trained clinicians . Using the SCQ in combination with other measures for internalizing and externalizing psychopathology is recommended [40,41,42,43].
Another notable finding was that the false negative group appeared to display fewer developmental and adaptive impairments compared to the true positive group, suggesting that the accuracy of the SCQ may be highly influenced by developmental factors. Children in the false negative group clearly exhibited ASD symptoms, and their ADOS-2 scores did not differ from those of the true positive group, but their presentations may have been subtler than those accurately captured with ASD by the SCQ or may be masked by their better-developed adaptive skills. Therefore, their caregivers were not able to reflect their observations on the SCQ. Our results corroborate previous findings that the SCQ may miss individuals with less overall impairment . Moreover, as a parent-report questionnaire, the SCQ highly relies on the reporter’s knowledge, experiences, observation, and understanding of autistic characteristics. Previous studies [44, 45] suggested that if a parent already has a prior understanding of ASD or knows their child has ASD, they are more likely to differentiate core autism symptoms from other challenging behaviors on questionnaires or endorse ASD symptoms to a greater degree, revealing the reporters’ knowledge influences how they respond to the screening questions. Parental concerns about their children’s development have also been suggested to influence parent-report instrument performance . It is possible that the SCQ could vary according to parental understanding of ASD and level of concern. Future research is needed to determine the optimal ways to screen for ASD in children where parents have minimal concerns about their children’s development.
Previous studies utilizing clinic-referred samples have suggested different cut-off scores according to age and purpose to increase accuracy [5, 10, 14]. Consistent with the literature, the current findings revealed suboptimal sensitivity and specificity of the SCQ, lower than those reported in the original study  as well as subsequent studies [5, 14]. However, adjusting cut-off scores based on age and sex did not improve the accuracy of the SCQ to a satisfactory level or impact the specificity of the measure on this sample. Our findings suggest that age and sex may not be the critical factors that impact the accuracy and usefulness of the SCQ among clinic-referred samples. Given our results suggesting suboptimal sensitivity and specificity of the SCQ, it is important to highlight for clinicians and researchers that the SCQ has limitations in accurately identifying children who will go on to receive an ASD diagnosis, particularly for children with complex presentations and other co-occurring mental health and behavioral diagnoses. Therefore, the SCQ used in isolation seems to have limited utility in assisting with the clinical triaging process or serving as a screening tool for inclusion or exclusion criteria in the research setting. This is problematic, given that there is a severe shortage of professionals who are trained to provide specialized evaluations for ASD, leaving specialty ASD clinics with long waiting lists, barriers to intervention access, and increased parental stress [47,48,49]. Tools that are validated in accurately classifying young children are critically necessary [50,51,52].
Several limitations of the present study should be addressed in future research. Although we had a relatively large sample, especially for a clinic-referred population that underwent gold-standard measures for ASD diagnosis, our sample sizes were made smaller when stratifying by age, sex, and SCQ status. In addition, selection bias could potentially limit the generalizability of our results. We were unable to explore additional family factors that may contribute to the sensitivity and specificity of the SCQ (e.g., primary language, race/ethnicity, and educational level) due to the limited inclusion of diverse families within our sample and inconsistent demographic information provided by families during the clinical intake process. It is also important to note that the SCQ score was available for the clinicians at the time of making a clinical diagnosis, and the clinical impression and diagnostic classification may not be independent. Further, the comorbidity of this clinic-referred sample was relatively high across the groups. It suggested that this is a complex sample, which might be contributing to the modest classification accuracy of the SCQ. The results may not generalize to other clinically referred samples with different clinical features. Future work should replicate our results in larger samples and explore additional factors that may contribute to individual differences related to the sensitivity and specificity of the SCQ.
In summary, the present study examined the sensitivity and specificity of the SCQ by assessing a clinical sample of children and adolescents with and without ASD. This study further examined the individual differences in sensitivity and specificity by describing the characteristics of children misclassified using the SCQ. These findings suggested that the SCQ may be less useful for young children with internalizing and externalizing symptoms and other more complex clinical presentations, highlighting the importance of taking an individual difference and multi-informant approach for ASD assessment. Clinically, these results underscore the necessity of comprehensive evaluations for children, particularly for internalizing and externalizing psychopathology in addition to autistic symptoms, to promote optimal diagnostic and intervention outcomes. In research settings, our results suggest that solely relying on the SCQ for inclusion criteria may be risky, especially for families who are less familiar with ASD. Future studies will be needed to determine if SCQ performs best in population-based screening (e.g., school-based screening) or clinically referred samples, as well as whether parental awareness of ASD impacts the reporting pattern on the SCQ, further influencing the sensitivity and specificity of the SCQ within certain populations.
Our findings make an important contribution to the literature on ASD measurement by specifying the behavioral characteristics of children misclassified by the SCQ. Specifically, individuals who were falsely classified as ASD by the SCQ exhibited more externalizing (e.g., aggression and conduct problems) and internalizing issues (e.g., anxiety), and they were more likely to be diagnosed with behavioral disorders (e.g., oppositional defiant disorder and conduct disorder) compared to other groups. The false negative group appeared to display fewer developmental and adaptive impairments compared to the true positive group. These findings suggest that the accuracy of the SCQ is highly influenced by developmental and behavioral factors, and the SCQ may be less useful for young children with internalizing and externalizing symptoms and other more complex clinical presentations. Clinicians and researchers could consider using the SCQ in combination with other assessment tools for internalizing and externalizing psychopathology.
Availability of data and materials
The datasets and unpublished materials used and/or analyzed in this study are available from the corresponding author on reasonable request.
Autism spectrum disorder
Attention-deficit hyperactivity disorder
Autism Diagnostic Observation Schedule, Second Edition
Autism Diagnostic Interview, Revised
Full Scale Intelligence Quotient
Behavioral Assessment System for Children, Parent Rating Scale
Vineland Adaptive Behavior Scale
Area under the receiver operating characteristic curve
Receiver operating characteristic
Restricted and Repetitive Behavior
Calibrated Severity Scores
Rutter M, Bailey A, Lord C. The social communication questionnaire. Los Angeles: Western Psychological Services; 2003.
Berument SK, Rutter M, Lord C, Pickles A, Bailey A. Autism screening questionnaire: diagnostic validity. Br J Psychiatry. 1999;175:444–51. https://doi.org/10.1192/bjp.175.5.444.
Hollocks MJ, Casson R, White C, Dobson J, Beazley P, Humphrey A. Brief report: An evaluation of the social communication questionnaire as a screening tool for autism spectrum disorder in young people referred to child & adolescent mental health services. J Autism Dev Disord. 2019;49(6):2618–23. https://doi.org/10.1007/s10803-019-03982-6.
Allen CW, Silove N, Williams K, Hutchins P. Validity of the social communication questionnaire in assessing risk of autism in preschool children with developmental problems. J Autism Dev Disord. 2007;37(7):1272–8. https://doi.org/10.1007/s10803-006-0279-7.
Corsello C, Hus V, Pickles A, Risi S, Cook EH Jr, Leventhal BL, et al. Between a ROC and a hard place: decision making and making decisions about using the SCQ. J Child Psychol Psychiatry. 2007;48(9):932–40. https://doi.org/10.1111/j.1469-7610.2007.01762.x.
Eaves LC, Wingert H, Ho HH. Screening for autism: agreement with diagnosis. Autism. 2006;10(3):229–42. https://doi.org/10.1177/1362361306063288.
Eaves LC, Wingert HD, Ho HH, Mickelson EC. Screening for autism spectrum disorders with the social communication questionnaire. J Dev Behav Pediatr. 2006;27(2 Suppl):S95–103. https://doi.org/10.1097/00004703-200604002-00007.
Norris M, Lecavalier L. Screening accuracy of Level 2 autism spectrum disorder rating scales. A review of selected instruments. Autism. 2010;14(4):263–84. https://doi.org/10.1177/1362361309348071.
Snow AV, Lecavalier L. Sensitivity and specificity of the Modified Checklist for Autism in Toddlers and the Social Communication Questionnaire in preschoolers suspected of having pervasive developmental disorders. Autism. 2008;12(6):627–44. https://doi.org/10.1177/1362361308097116.
Chesnut SR, Wei T, Barnard-Brak L, Richman DM. A meta-analysis of the social communication questionnaire: screening for autism spectrum disorder. Autism. 2017;21(8):920–8. https://doi.org/10.1177/1362361316660065.
Barnard-Brak L, Brewer A, Chesnut S, Richman D, Schaeffer AM. The sensitivity and specificity of the social communication questionnaire for autism spectrum with respect to age. Autism Res. 2016;9(8):838–45. https://doi.org/10.1002/aur.1584.
Miranda A, Berenguer C, Rosello B, Baixauli I. Relationships between the social communication questionnaire and pragmatic language, socialization skills, and behavioral problems in children with autism spectrum disorders. Appl Neuropsychol Child. 2020;9(2):141–52. https://doi.org/10.1080/21622965.2018.1550403.
Moody EJ, Reyes N, Ledbetter C, Wiggins L, DiGuiseppi C, Alexander A, et al. Screening for autism with the SRS and SCQ: variations across demographic, developmental and behavioral factors in preschool children. J Autism Dev Disord. 2017;47(11):3550–61. https://doi.org/10.1007/s10803-017-3255-5.
Chandler S, Charman T, Baird G, Simonoff E, Loucas T, Meldrum D, et al. Validation of the social communication questionnaire in a population cohort of children with autism spectrum disorders. J Am Acad Child Adolesc Psychiatry. 2007;46(10):1324–32. https://doi.org/10.1097/chi.0b013e31812f7d8d.
Rosenberg SA, Moody EJ, Lee LC, DiGuiseppi C, Windham GC, Wiggins LD, et al. Influence of family demographic factors on social communication questionnaire scores. Autism Res. 2018;11(5):695–706. https://doi.org/10.1002/aur.1935.
Wiggins LD, Bakeman R, Adamson L, Robins DL. The Utility of the Social Communication Questionnaire in screening for autism in children referred for early intervention. Focus Autism Other Dev Disabil. 2007;22(1):33–8.
Wiggins LD, Reynolds A, Rice CE, Moody EJ, Bernal P, Blaskey L, et al. Using standardized diagnostic instruments to classify children with autism in the study to explore early development. J Autism Dev Disord. 2015;45(5):1271–80. https://doi.org/10.1007/s10803-014-2287-3.
Frazier TW, Georgiades S, Bishop SL, Hardan AY. Behavioral and cognitive characteristics of females and males with autism in the Simons Simplex Collection. J Am Acad Child Adolesc Psychiatry. 2014;53(3):329-40.e1-3. https://doi.org/10.1016/j.jaac.2013.12.004.
Kaat AJ, Shui AM, Ghods SS, Farmer CA, Esler AN, Thurm A, et al. Sex differences in scores on standardized measures of autism symptoms: a multisite integrative data analysis. J Child Psychol Psychiatry. 2020. https://doi.org/10.1111/jcpp.13242.
Evans SC, Boan AD, Bradley C, Carpenter LA. Sex/gender differences in screening for autism spectrum disorder: implications for evidence-based assessment. J Clin Child Adolesc Psychol. 2019;48(6):840–54. https://doi.org/10.1080/15374416.2018.1437734.
Ghaziuddin M, Welch K, Mohiuddin S, Lagrou R, Ghaziuddin N. Utility of the Social and Communication Questionnaire in the differentiation of autism from ADHD. J Dev Phys Disabil. 2010;22(4):359–66. https://doi.org/10.1007/s10882-010-9199-8.
Marvin AR, Marvin DJ, Lipkin PH, Law JK. Analysis of Social Communication Questionnaire (SCQ) screening for children less than age 4. Curr Dev Disord Rep. 2017;4(4):137–44. https://doi.org/10.1007/s40474-017-0122-1.
Suren P, Saasen-Havdahl A, Bresnahan M, Hirtz D, Hornig M, Lord C, et al. Sensitivity and specificity of early screening for autism. BJPsych Open. 2019;5(3):e41. https://doi.org/10.1192/bjo.2019.34.
Brooks WT, Benson BA. The validity of the social communication questionnaire in adults with intellectual disability. Res Autism Spectr Disord. 2013;7(2):247–55.
Sappok T, Diefenbacher A, Gaul I, Bolte S. Validity of the social communication questionnaire in adults with intellectual disabilities and suspected autism spectrum disorder. Am J Intellect Dev Disabil. 2015;120(3):203–14. https://doi.org/10.1352/1944-7558-120.3.203.
Ung D, Johnco C, McBride NM, Howie F, Scalli L, Storch EA. Optimizing the screening of autism spectrum disorders in outpatient clinics: an examination of the Social Communication Questionnaire-Lifetime. Res Autism Spectr Disord. 2016;27:21–8.
Witwer AN, Lecavalier L. Autism screening tools: an evaluation of the social communication questionnaire and the developmental behaviour checklist-autism screening algorithm. J Intellect Dev Disabil. 2007;32(3):179–87. https://doi.org/10.1080/13668250701604776.
Mouti A, Dryer R, Kohn M. Differentiating autism spectrum disorder from ADHD using the social communication questionnaire. J Atten Disord. 2019;23(8):828–37. https://doi.org/10.1177/1087054718781945.
Schwenck C, Freitag CM. Differentiation between attention-deficit/hyperactivity disorder and autism spectrum disorder by the social communication questionnaire. Atten Defic Hyperact Disord. 2014;6(3):221–9. https://doi.org/10.1007/s12402-014-0147-9.
Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994;24(5):659–85. https://doi.org/10.1007/BF02172145.
Reynolds CR, Kamphaus RW. Behavior assessment system for children (3rd ed.). Bloomington: Pearson; 2015.
Sparrow SS, Cicchetti DV, Balla AD. Vineland adaptive behavior scales, second edition (Vineland-II). Circle Pines: American Guidance Service; 2005.
Sparrow SS, Cicchetti DV, Saulnier CA. Vineland adaptive behavior scales, third edition (Vineland-3). San Antonio: Pearson; 2016.
Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop S. Autism diagnostic observation schedule, second edition (ADOS-2). Torrance: Western Psychological Services; 2012.
Gotham K, Pickles A, Lord C. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. J Autism Dev Disord. 2009;39(5):693–705. https://doi.org/10.1007/s10803-008-0674-3.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.
Ben-Sasson A, Cermak SA, Orsmond GI, Tager-Flusberg H, Kadlec MB, Carter AS. Sensory clusters of toddlers with autism spectrum disorders: differences in affective symptoms. J Child Psychol Psychiatry. 2008;49(8):817–25. https://doi.org/10.1111/j.1469-7610.2008.01899.x.
Adams D, Young K, Simpson K, Keen D. Parent descriptions of the presentation and management of anxiousness in children on the autism spectrum. Autism. 2019;23(4):980–92. https://doi.org/10.1177/1362361318794031.
Molloy CA, Murray DS, Akers R, Mitchell T, Manning-Courtney P. Use of the Autism Diagnostic Observation Schedule (ADOS) in a clinical setting. Autism. 2011;15(2):143–62. https://doi.org/10.1177/1362361310379241.
Heavner K, Newschaffer C, Hertz-Picciotto I, Bennett D, Burstyn I. Quantifying the potential impact of measurement error in an investigation of autism spectrum disorder (ASD). J Epidemiol Community Health. 2014;68(5):438–45. https://doi.org/10.1136/jech-2013-202982.
Hossain MM, Khan N, Sultana A, Ma P, McKyer ELJ, Ahmed HU, et al. Prevalence of comorbid psychiatric disorders among people with autism spectrum disorder: an umbrella review of systematic reviews and meta-analyses. Psychiatry Res. 2020;287:112922. https://doi.org/10.1016/j.psychres.2020.112922.
Johnson S, Hollis C, Hennessy E, Kochhar P, Wolke D, Marlow N. Screening for autism in preterm children: diagnostic utility of the Social Communication Questionnaire. Arch Dis Child. 2011;96(1):73–7. https://doi.org/10.1136/adc.2010.194795.
Lord C, Charman T, Havdahl A, Carbone P, Anagnostou E, Boyd B, et al. The Lancet Commission on the future of care and clinical research in autism. Lancet. 2022;399(10321):271–334.
Miller L, Burke J, Robins D, Fein D. Diagnosing autism spectrum disorders in children with low mental age. J Autism Dev Disord. 2019;49(3):3550–61.
Christopher K, Bishop S, Carpenter LA, Warren Z, Kanne S. The implications of parent-reported emotional and behavioral problems on the modified checklist for autism in toddlers. J Autism Dev Disord. 2021;51:884–91.
Havdahl KA, Bishop SL, Suren P, Oyen AS, Lord C, Pickles A, et al. The influence of parental concern on the utility of autism diagnostic instruments. Autism Res. 2017;10(10):1672–86. https://doi.org/10.1002/aur.1817.
Martinez M, Thomas KC, Williams CS, Christian R, Crais E, Pretzel R, et al. Family experiences with the diagnosis of autism spectrum disorder: system barriers and facilitators of efficient diagnosis. J Autism Dev Disord. 2018;48(7):2368–78. https://doi.org/10.1007/s10803-018-3493-1.
Mazurek MO, Curran A, Burnette C, Sohl K. ECHO autism STAT: accelerating early access to autism diagnosis. J Autism Dev Disord. 2019;49(1):127–37. https://doi.org/10.1007/s10803-018-3696-5.
Osborne LA, Reed P. Parents’ perceptions of communication with professionals during the diagnosis of autism. Autism. 2008;12(3):309–24. https://doi.org/10.1177/1362361307089517.
Robins DL, Adamson LB, Barton M, Connell JE Jr, Dumont-Mathieu T, Dworkin PH, et al. Universal autism screening for toddlers: recommendations at odds. J Autism Dev Disord. 2016;46(5):1880–2. https://doi.org/10.1007/s10803-016-2697-5.
Siu AL, Force USPST, Bibbins-Domingo K, Grossman DC, Baumann LC, Davidson KW, et al. Screening for autism spectrum disorder in young children: US preventive services task force recommendation statement. JAMA. 2016;315(7):691–6. https://doi.org/10.1001/jama.2016.0018.
Towle PO, Patrick PA. Autism spectrum disorder screening instruments for very young children: a systematic review. Autism Res Treat. 2016;2016:4624829. https://doi.org/10.1155/2016/4624829.
We extend our heartfelt appreciation to the participating families whose time and effort made this study possible. Their contribution has been invaluable, and their willingness to participate is essential to advancing research in this field.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program (NSF-GRF 00074041) to M. Altschuler. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from parents of all individual participants included in the study, and all children provided their assent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lee, C.M., Altschuler, M.R., Esler, A.N. et al. Why are only some children with autism spectrum disorder misclassified by the social communication questionnaire? An empirical investigation of individual differences in sensitivity and specificity in a clinic-referred sample. J Neurodevelop Disord 15, 28 (2023). https://doi.org/10.1186/s11689-023-09497-7