The study was approved by the institutional review board at Rush University Medical Center (RUMC), and all participants or their parents signed informed consent according to an approved protocol.
Participants with FXS were recruited through the Fragile X Clinic and Research Program at RUMC, In total, 36 participants (28 male, 8 female; mean ± standard deviation (SD) 18.0 ± 10.24 years of age, range 7 to 50) with the full Fragile X mutation confirmed by standard testing using Southern blot combined with PCR. All participants were tested using the computer-based KiTAP test.
Subject participation was not limited by age, gender or IQ score because the intent of the study was to evaluate a wide range of subjects to determine the age and functional range over which the KiTAP could be used as an outcome measure in clinical trials with subjects with FXS. It was important to evaluate the KiTAP in both genders, and in both children and adults, given that trials of new targeted treatments for FXS would probably involve all these groups. However, all subjects tested were verbal and able to speak at least in phrases, and all had sufficient receptive language to follow basic directions.
When available, cognitive (IQ) assessments were obtained from previous tests (within the past 2 years for subjects under 15 years of age, and within the past 5 years for subjects over 15 years of age at the time of the IQ test) performed at RUMC or from psychological assessments at the participant's school. Subjects for whom IQ test results were not available were not precluded from participating in the study. IQ assessments represented scores from the Wechsler Intelligence Scale for Children (WISC), Wechsler Adult Intelligence Scale, or Stanford-Binet tests, and were available for 24 (17 males and 7 female) participants with FXS. MA was calculated from IQ using the formula
with a maximum value of 15 used for chronological age.
The comparison cohort, consisting of 25 individuals with ID (13 male and 12 female; mean ± SD 11.4 ± 4.4 years of age. range 5 to 24), was recruited and tested using identical protocols at the Medical Investigation of Neurodevelopmental Disorders (MIND) Institute at UC Davis. This group consisted of 12 subjects with idiopathic ID, 8 with Down syndrome, 2 with fetal alcohol syndrome, 1 with 17q21.31 deletion syndrome, 1 with 22q11.2 deletion syndrome, and 1 with ID and autism. IQ measures, performed with the WISC or Stanford-Binet tests, were available for all ID participants. Although the FXS and ID subjects were tested at different sites, the methods for administration of the KiTAP were standardized as much as possible through phone conferences and a face-to-face meeting. The ID group was examined to see if there were differences in performance patterns that might be attributed to the FXS phenotype alone rather than to general ID. For this purpose, retest was not necessary, and so the ID group was not retested.
Distribution of psychotropic medication use in the FXS group was as follows: no drug (14 subjects); one drug (10 subjects), two drugs (8 subjects), and three drugs (2 subjects). Medications used were stimulants (15 subjects), selective serotonin uptake inhibitors (SSRIs)/antidepressants (10 subjects), antipsychotics (4 subjects), and lithium (1 subject).. Distribution of psychotropic medication use in the ID group was: no drug (18 subjects), one drug (3 subjects), two drugs (3 subjects), and three drugs (1 subject). Medications used were stimulants (2 subjects), SSRIs/antidepressants (2 subjects), antipsychotics (4 subjects), valproic acid (1 subject), and clonidine (2 subjects).
The KiTAP is composed of eight subtests designed around a theme (an enchanted castle) specifically designed to be accessible to young children, thus providing more motivation to sustain interest in testing than CPTs and other EF batteries based on abstract symbols. Each subtest measures a different aspect of cognition: alertness, distractibility, divided attention, flexibility, reaction control (inhibition), sustained attention, vigilance, and visual scanning. The Alertness subtest (The Witch) requires subjects to tap a button every time a stimulus (a witch) appears on the screen. The Distractibility subtest (The Happy and Sad Ghosts) requires subjects to tap a button when a target stimulus (a sad ghost) appears on the screen while ignoring distracters that appear shortly before the stimulus; subjects are supposed to 'cheer up' the sad ghost by pushing the button and must not respond when the happy ghost because it does not need cheering up. The Flexibility subtest (The Dragons' House) requires subjects to alternate between identifying blue and green dragons which seem on random sides of the screen by tapping one of two buttons. The Go/NoGo or inhibition subtest (The Bat and the Cat) requires subjects to tap a button when the target stimulus (a bat) is presented, while refraining from hitting the button for the non-target stimulus (a cat). The Visual Scanning subtest (The Witches' Parade) requires subjects to scan a grid of 25 witches; subjects press one button if all witches are flying in the same direction, and hit a second button if one of the witches is flying in the opposite direction. The Vigilance subtest (The Mirror) is similar to Go-NoGo, but the target stimulus (a ghost with orange eyes) appears infrequently, and the test is much longer (approximately 10 minutes). The Sustained Attention subtest (The Ghost's Ball) features a sequence of different-colored ghosts; subjects press a button when two ghosts of the same color appear sequentially. The Divided Attention subtest (The Owl) is the only task requiring processing of an auditory stimulus in addition to visual stimuli, and requires subjects to simultaneously listen to a series of high and low owl sounds and watch for target stimuli (owls with closed eyes). Subjects must press a button either when a sound is repeated or when the target stimulus appears.
Subjects spent up to 90 minutes completing all KiTAP subtests if they were able to perform the entire test (only five subjects). Most subjects spent about 30 minutes doing the test, in which they completed 4 to 5 subtests, but were unable to focus upon, or did not understand what to do in subsequent longer subtests. Subtests were administered in the order: Alertness, Distractibility, Flexibility, Go/NoGo, Visual Scanning, Vigilance, Sustained Attention, and Divided Attention. Because the last four are much longer tasks, and based on previous experience of the length of task that typically functioning individuals with FXS can tolerate, a decision was made to begin with the subtests most likely to be completed by a wide range of participants, and to leave the most difficult and longest subtests (Sustained Attention and Divided Attention) until last. We continued to administer subtests of increasing length and difficulty until the subject refused to do any more, until it became clear to the examiner that the participant was not making any real attempt to perform the task, or until the subject was not able to show that they understood the sample pretest. Although there is the possibility that this testing order may have introduced bias into the analysis of the ability of subjects with FXS to complete the subtests, the testing order was necessary, based on initial experience with pilot subjects, as administering the longer and more difficult tests first was likely to lead to refusal, thus precluding later administration of the tests the subjects would be most likely able to complete. The order of test administration was therefore driven predominantly by the length of the task and the restrictions posed by the limited attention of subjects with FXS. Further, it is important in clinical trials or for interventions in the clinic to use the same test order before and after administration of the intervention, and the most important focus of this study was to determine the reliability of the KiTAP subtests for intervention and longitudinal studies that would involve multiple testing sessions.
During the tests, subjects were aided by a research assistant (RA). Before the start of each subtest, the RA explained the goal of the test. The subject then took a short pretest to allow their understanding to be evaluated, and if necessary repeated the pretest after further explanation, up to a maximum of three pretests. If the subject was ultimately able to perform the pretest for a given subtest correctly, and the RA perceived that the subject understood the test, the subtest was then administered. If the subject could not show that they understood what to do on the subtest, the subtest was listed as 'not feasible' for that subject. The RA also encouraged subjects to continue working, and gave positive reinforcement about performance during subtests, provided breaks between subtests when necessary, and dismissed subjects when they finished all subtests or were judged unwilling or unable to continue testing. The administration method was consistent between both centers (Rush and UC Davis), including the scripts used to explain tests to subjects and the prompting strategies. The scripts used to explain the task for the pretest were those in the KiTAP instruction manual, but these were repeated and rephrased in simpler language (same content) when subjects did not understand and were unable to perform the pretest correctly. Prompts to encourage the subjects to continue working on the task were given when they stopped paying any attention to the screen and started to do something else or were clearly no longer engaged in the testing (for example, 'Watch the computer, we have to find the witches.'). Subjects were given non-specific positive feedback for continuing to work regardless of whether they were giving accurate responses or not (for example, 'You're doing a really good job with this test today.'). Subjects were not helped with the test or told to push the button when stimuli appeared. As would be expected, the number of prompts required varied depending on the subject.
Of the 36 subjects with FXS who were tested with the KiTAP, 29 were retested 2 to 3 weeks later by the same tester in the same setting. The other seven subjects (five male, two females; mean ± SD 18.3 ± 12.4 years of age, range 9 to 44) were not retested because of scheduling issues. No changes in psychotropic medication were allowed between testing sessions. IQ data were available for 22 (16 male and 6 female) subjects who completed retesting.
Behavioral validation measures
At the initial session, parents of all participants, including adults who consented for themselves, filled out both the Behavior Assessment System for Children-Second Edition (BASC-2)  and Aberrant Behavior Checklist-Community Edition (ABC-C)  to allow correlation of performance on the KiTAP with ratings of each subject's hyperactivity and attentional function in daily life. The ABC-C is the most widely used scale to quantify behavioral symptoms in individuals with ID including FXS [17, 28], and is considered the most valid currently available scale to assess hyperactivity and impulsive behavior for a cohort of subjects with FXS. The ABC-C is a 58-item parent- or caregiver-rated scale designed to assess adaptive and maladaptive behavior of intellectually disabled people. It is divided into five subscales: Irritability (15 items), Lethargy/Social Withdrawal (16 items), Stereotypic Behavior (7 items), Hyperactivity (16 items), and Inappropriate Speech (4 items). The BASC-2 was used to supplement the ABC-C data, as the ABC-C does not specifically cover attention. The BASC-2 is a comprehensive set of behavior-rating scales measuring degree of clinically relevant problems, including aggression, anxiety, Attention Problems, atypicality, conduct problems, hyperactivity, depression, somatization, and withdrawal. All items are rated on a three-point scale. There is a parent-rated version for children aged 6 to 11 years and adolescents aged 12 to 21 years, and a self-rated version for adults over the age of 21 years. The attention and hyperactivity clinical scales were chosen for analysis as they were deemed most relevant to the dimensions of Attention and inhibitory control on the KiTAP. The Adaptability scale was also chosen for analysis, as it appeared most likely to show association with the flexibility construct of the KiTAP. There are no scales addressing attention that have been specifically validated for FXS. The BASC-2 was chosen because it has been extensively used to rate attention and hyperactivity in children with complex behavior disorders, and has been used in practice by our groups to evaluate attention and distractibility in patients with FXS, particularly for females and higher-functioning males. The children's version of the BASC-2 was used for rating individuals with FXS aged 6 to 11 years and the adolescent version for individuals aged 12 years or older, including adults (there were only three individuals in the study with FXS over the age of 22 years). The adolescent version was used for the adults with FXS (who have substantially lower MAs) because these individuals are unable to rate themselves (the adult form is self-rated), and the questions on the adolescent version were more appropriate for these individuals, whereas the items on the adult form are largely not relevant to adults with FXS.
Data analysis was performed using a spreadsheet (Excel; Microsoft Corp., Redmond, WA, USA and SPSS software (IBM, Armonk, NY, USA). Several measures from each KiTAP subtest were analyzed including number correct, errors, omissions, median reaction time, and SD of median reaction time. Raw scores were used because the normalized scores on the KiTAP previously generated from a typical population of children would not be relevant to individuals with FXS. For each of these measures, the distribution of the scores was plotted, and the intraclass correlation coefficient (ICC) was computed in SPSS for measures that had an acceptable distribution (normal distribution, and no ceiling, floor, or learning effects) after outliers and non-valid data points (subject not really participating) were eliminated. To examine clinical relevance, Pearson correlations between KiTAP scores and eight behavior checklist scores (five subscale scores from the ABC-C and the three subscale T-scores from the BASC-2) were calculated. For all subjects with FXS and ID who had completed IQ testing, MA values from the assessments were used as estimates of cognitive level to allow determination of the minimum ability required for valid testing in both children and adults (for example, a given IQ in a child and adult will not represent a comparable functional level) and to allow MA-matching in the FXS/ID comparisons. Correlations between MA and scores on KiTAP measures shown to be valid and reliable were calculated. KiTAP subtest scores in FXS and ID groups were compared using the t-test. Because this was a pilot study, significance was set at P = 0.05 for all comparisons, without adjustment for multiple comparisons.