Feasibility, reliability, and clinical validity of the Test of Attentional Performance for Children (KiTAP) in Fragile X syndrome (FXS)

Fragile X syndrome (FXS) is the most common known inherited cause of intellectual disability (ID), learning disability, and autism, with an estimated frequency in the range of about 1:2500 to 1:4000 [1]. FXS results from a trinucleotide repeat (CGG) expansion mutation of more than 200 repeats (full mutation) in the promoter of FMR1 (Fragile X mental retardation 1 gene), which leads to transcriptional silencing of FMR1, and loss or significant reduction of expression of the gene product, the Fragile X mental retardation protein (FMRP). FMRP is an RNA-binding protein that acts as a negative modulator of dendritic translation. Loss of FMRP results in excessive and dysregulated dendritic translation, producing aberrant dendritic morphology and synaptic plasticity, and leading to abnormal development and cognition [2–4]. In addition to intellectual disability, the resultant behavioral phenotype is characterized by prominent deficits in attention and inhibitory control; autistic symptoms including social and communication deficits, and stereotypic behavior; social anxiety and withdrawal; hyperarousal; sensory defensiveness; and gaze aversion [5, 6].

Recent advances in the neurobiology of FXS have suggested that many of the phenotypic features of the disorder arise from enhanced activity of translational activation pathways regulated by metabotropic glutamate receptors 1 and 5 (mGluR1, mGluR5), as a result of absence of FMRP [7]. In support of this, most known phenotypes in the mouse and fly models of FXS (which lack FMRP) can be reversed by pharmacologically [8] or genetically [9] downregulating these pathways. These groundbreaking studies have set the stage for pharmacological trials in humans with FXS, designed to target the excess activity in mGluR-regulated translational pathways in neurons [10].

Thus, there is an urgent need to develop objective and well-validated outcome measures that assay core FXS phenotypes. Boys with FXS show larger attention and executive function (EF) deficits than do mental age (MA)-matched boys with Down syndrome or typically developing boys, particularly in areas involving switching attention and inhibiting repetitious behavior [11]. Thus, EF and inhibition problems such as hyperactivity, impulsiveness, and distractibility are thought to be core features of FXS, occurring in about 80 to 90% of males and at least half of females in survey studies [12, 13]. Behavior-rating scales, regarded as the standard means of assessing these symptoms, are subject to problems of rater bias and placebo effects. Continuous performance tasks (CPT) can more objectively measure EFs, and have the benefit of being responsive to relatively short-term medication treatments, making them a potentially useful type of efficacy test for early phase clinical trials and for testing response to interventions in clinic [14–16]. Efforts to use CPTs to measure function in these areas have been problematic in FXS because of the cognitive impairment and widely varying levels of function. Previous CPTs used in medication trials in FXS have been too difficult for the majority of participants (for example, the Integrated Visual and Auditory (IVA) CPT [17]) or too easy for the higher-functioning and adult participants, resulting in a high frequency of ceiling scores (for example, the North Carolina Fragile X Project CPT (FXCPT)[18] Clinical experience of patients with FXS suggests that standard CPTs used in clinical practice, such as the Connors CPT, tend to be too difficult or too long for many individuals with FXS, thus meaningful data cannot be obtained, or the individual loses interest and does not complete the task. Most CPTs used in clinical practice such as the IVA+Plus (BrainTrain, Richmond, VA, USA) are validated with a lower age limit of 5 to 6 years. Because the average MA of adult males with FXS is 5 to 6 years [19], these tasks lack the range to accurately measure the abilities of at least half of adult males with FXS and of less than half of younger males who have a lower average MA. Many of these CPTs are not interesting to the participants, presenting just simple numerals for subject responses, and are administered over a longer time period than most males with FXS can tolerate. Although the tests are designed to be uninteresting in order to test attention focusing over time in the general population, a large fraction of the FXS population, in whom ability to maintain attention and EF is much more impaired, will often discontinue the task after a short period of cooperation. Refusal to perform tasks perceived as difficult or long is a major problem in testing of individuals with FXS [20], and it is documented that boys with FXS have difficulty completing executive-function tasks [21]. By contrast, using a simplified test with limited range such as the North Carolina FX-CPT, may result in higher-functioning individuals with FXS showing only ceiling scores [21].

The Test of Attentional Performance for Children (Testbatterie zur Aufmerksamkeitsprüfung für Kinder; KiTAP) is a computer-based CPT and EF battery [22] that has not previously been used to measure cognitive function in subjects with FXS. The KiTAP was adapted for use in children from the Tests of Attentional Performance (TAP), a test used since the late 1990s to measure attention and EF performance in adults with various medical, neurological, and psychiatric conditions. KiTAP and TAP have been translated from the original German into validated versions in English, French, Spanish, and Italian [23]. The KiTAP was devised to be interesting to children, with an enchanted-castle theme as opposed to abstract stimuli, and has been used to show attentional impairment, with slower reaction times and increases in errors and omissions correlating with decrements in intelligence quotient (IQ) in liver-transplanted 6 to 12-year-old children [24], increased impulsivity (false alarms) in lead-exposed 8 to 12-year-old children that are more evident than IQ decrements [25], and improvement in Sustained Attention measures after administration of methylphenidate to 5 to 12-year-old children with attention deficit hyperactivity disorder [16]. The KiTAP is composed of eight tests that vary in length and difficulty level, thus it may be possible to use this test to measure attentional function over the broad range of intellectual ability present in a cohort with a disorder such as FXS. Other reasons for choosing the KiTAP to study FXS are the visual nature of the test, given the high preference for visual stimuli seen for subjects with FXS, and the identifiable and high-interest 'characters' presented in the enchanted-castle theme.

In this study, we sought to determine the feasibility (level of function required to achieve scores on different subtests without ceiling or basal effects), reproducibility (consistency of scores in a test/retest setting), and validity (correlation with meaningful measures of maladaptive behavior) of the subtests of the KiTAP in a population with FXS. Additionally, scores were compared with those of MA controls with ID as a means of evaluating specificity of performance patterns on the KiTAP in FXS.

The study was approved by the institutional review board at Rush University Medical Center (RUMC), and all participants or their parents signed informed consent according to an approved protocol.

Basal and ceiling effects were related to MA for the subjects with FXS for whom we had valid IQ scores (Table 2). Basal effects resulted predominantly from unwillingness to complete the test or inability to engage in the test with meaningful effort, and a ceiling effect resulted from perfect scores in number correct, errors, or omissions. The Alertness subtest was completed by all participants, but ceiling effects were found for all measures (aside from reaction time) for some subjects with MA above 6.3 years and for all subjects with MA above 7.8 years. For the Distractibility and Go/NoGo subtests, some failures were found for subjects with MA below 5.4 years, and a ceiling effect for some subjects with MA above 7.1 and 7.0 years, respectively. Flexibility was completed by all subjects whose MA was known, and did not show a consistent ceiling effect with MA. For the Vigilance and Visual Scanning subtests, failures were found when MA was lower than 7.5 years. There was also a ceiling effect for Vigilance when MA was higher than 7.13 years, but no ceiling effect for Visual Scanning. The Sustained Attention and Divided Attention subtest could not be completed by the majority of the subjects, but no ceiling effects were seen for subjects who did attempt the tests.

In this study, we sought to determine the feasibility, reliability, and validity of the KiTAP in FXS. Our data demonstrates that several measures from the KiTAP show an adequate range and good feasibility, are relatively stable over time, and are correlated with clinically meaningful behavioral constructs. The KiTAP subtests had different ranges of MA for which there were no ceiling, basal, or learning effects, and thus subjects of all MAs received a valid score on at least some of the subtests. In general, however, a significant minority of subjects with FXS could not perform the Vigilance and Visual Scanning subtests, and the large majority could not perform the Sustained Attention and Divided Attention subtests; the former because of the subjects' inability to maintain attention and motivation long enough to complete the task, and the latter because of the high level of task complexity and the subjects' inability to remember directions for both auditory and visual stimuli

Each of the subtests that were feasible for the majority of subjects with FXS had specific measures with an acceptable range and distribution of scores, and good test/retest reproducibility (ICC < 0.6). Finally, each of these feasible and reproducible measures showed some correlation with clinical behaviors, with the exception of Go-NoGo reaction time, and these correlations were mostly in the expected direction with respect to the clinical behaviors. For example, subjects who scored higher in Hyperactivity on ABC-C and BASC-2 tended to commit more impulsive errors, and subjects who scored worse on Adaptability tended to commit more errors on the Flexibility subtest. However, reaction time data was more difficult to interpret. Indeed subjects showing shorter reaction times on flexibility had higher hyperactivity ratings. This may reflect a relationship between more impulsive responding (and therefore shorter reaction time) and hyperactivity on the BASC-2, although other reaction time scores on KiTAP subtests seemed to be positively correlated with hyperactivity, suggesting that hyperactivity may result in more off-task behavior and slower reaction time. These results are difficult to reconcile, but may be just an effect of small sample size that do not reflect a clinical pattern. Our data suggest that some scores on the KiTAP may reflect the severity of aspects of the FXS phenotype, but this will require more study in a larger sample.

It is difficult to know in the FXS population whether the subtests of the KiTAP were specifically measuring the EFs they were designed to measure. Despite the correlations with the behavioral forms, which represent a crude screen for clinical relevance, there is no validated EF measure or CPT for cohorts with FXS, and therefore there is no standard for comparison to evaluate the specificity of the subtests in FXS. Previously, CPTs have been used in FXS cohorts in a clinical trial setting, and to measure medication effects on EF and attention in patients treated in clinical practice. When administered to 48 adults with FXS in a clinical trial, the IVA produced data that were impossible to analyze [17]. In this studythe combination of auditory and visual information appeared too complex for individuals with FXS to analyze, and the abstract, uninteresting nature of the stimuli failed to hold the subject's attention, resulting in random responding. The North Carolina project CPT has been subjected to some psychometric study, and was used to track development of EF and to evaluate effects of clinical stimulant treatment on attention trajectories [29] in young males with FXS, but proved to have insufficient range when used in a clinical trial with higher-functioning male and female adults with FXS, thus suggesting that it is useful only for a subset of young patients [18].

We found that many, but not all, KiTAP measures correlated with MA, both in the FXS group and the ID group. There was no association between chronological age and KiTAP performance in either group, and for the MA-matched FXS and ID groups used to compare the KiTAP subtest scores, there was no significant difference in chronological age, so age should not have been a factor limiting the comparisons. For most KiTAP measures there were no significant differences in score distributions between the two groups for the different subtests, suggesting that KiTAP gives a general measure of executive-function ability both in subjects with FXS and in those with other causes of cognitive impairment. The difference in flexibility scores, with worse performance in the FXS group, is not unexpected given that perseveration is a prominent feature of the FXS phenotype. However, it was unexpected that the FXS group made fewer errors on the Distractibility sub-test; this may be due to the high demand for rapid visual processing inherent in the test (stimuli are presented very rapidly) coupled with the relative strength in visual processing in FXS [30], or to some other property of the specific subtest, or to random chance, as the significance level was borderline.

Specific measures on the KiTAP that emerged as feasible for the majority of subjects were Alertness reaction time, SD of Alertness reaction time, Distractibility errors, Go/NoGo errors, Flexibility errors, Flexibility reaction time, and Vigilance reaction time (Table 5). These measures did not produce excessive ceiling, basal, or learning effects, exhibited an acceptable range and distribution of scores, had good to excellent reproducibility, and correlated with meaningful behavior ratings. Six of these seven measures were contained in four subtests: Alertness, Distractibility, Go/NoGo, and Flexibility. In our experience administering the KiTAP, we found that these tests also have the advantage of each requiring less than 5 minutes to administer, as opposed to other subtests, which required 10 to 15 minutes. This allowed the majority of subjects to complete all four of these subtests without significant fatigue, a definite advantage for assessments involving subjects with FXS. Thus, we feel that these four subtests are best suited for use in future studies, and because this pilot study has shown that most patients with FXS can complete these subtests, the effects of test order can be further evaluated by randomizing the order of administration in future studies. Further, more work could be carried out to evaluate the characteristics of the Vigilance, Visual Scanning, Sustained Attention, and Divided Attention Subtests in a larger high-functioning FXS sample with MA of greater than 10 years to determine whether these are useful measures of response in high-functioning FXS cohorts, particularly females.

In this pilot study, some measures on at least some subtests of the KiTAP showed adequate feasibility, reliability, and clinical validity for subjects with FXS across a wide range of function. This is in contrast with previous CPTs which have not allowed meaningful administration to such a range of individuals with FXS without producing substantial ceiling and floor effects [17, 18]. Given the difficulties with rater bias and placebo effects that accompany behavioral rating forms addressing EF domains even with optimal administration practices, the KiTAP may be a useful objective outcome measure of attention and executive function for use in FXS clinical trials, and even to track medication responses in clinical practice. Further work is needed to evaluate the responsiveness of the KiTAP to pharmaceutical or other interventions.