Introduction

The COVID-19 pandemic led to a significant shift in how society functions, with a major move from in person interactions to remote alternatives. This shift introduced several challenges, especially in healthcare settings, where hands on assessments were no longer possible. During this time, telemedicine became a key player in transforming the traditional models of healthcare delivery. However, this transition also posed several challenges, particularly in the area of sports and athlete assessments, including concussion baseline testing.

Baseline neurocognitive testing before the start of sports seasons is considered crucial for diagnosing and managing concussions by specialists in the field [1,2]. One of the most commonly used tools for this testing is the Immediate Post-Concussion Assessment and Cognitive Test (ImPACT) [3,4]. Traditionally, this baseline testing is often done in a group setting, such as a computer lab with up to 10 individuals, which offers time saving advantages for athletic teams and trainers. While group settings are efficient, they may introduce more distractions compared to individual testing, which, while quieter, may be perceived as more time consuming. There has been ongoing debate about the impact of the testing environment on the outcomes, particularly regarding group versus individual settings.

Research conducted by Moser and colleagues in 2011 suggested a significant difference in baseline performance based on the setting [5]. However, more recent studies by French, et al. and Vaughan, et al. indicated no significant difference in the cognitive performance between group and individual testing environments [6,7]. In addition to the setting, factors such as age and sex may also influence baseline neurocognitive performance, although studies on these variables have shown mixed results [8].

College athletes often juggle their neurocognitive baseline testing with a busy schedule that includes academics, practice sessions, training, and competitions. These athletes face stressors that their non-athletic peers typically do not experience, such as physical fatigue, mental pressures from their sport, and the demands of traveling and competing [9,10]. Given the extra demands on their time and energy, it can be challenging for athletes to find time for baseline screenings. If athletes were allowed to complete these assessments independently from home, it could save them valuable time and effort. Recently, the developers of the ImPACT test have made it possible to complete the test remotely using a unique, one-time access code. This raises the question of whether there are differences in performance when athletes take the ImPACT remotely at home versus in a controlled clinic setting.

Most of the existing research on the impact of testing environment on neurocognitive assessments has focused on younger populations, particularly adolescents, leaving a knowledge gap for collegiate athletes. Additionally, the influence of remote testing, especially in the context of the COVID-19 pandemic, remains largely unexplored. This shift to remote testing opens up new avenues for how neuropsychological assessments, such as concussion baseline testing, could be conducted in the future.

The ImPACT test, which is typically administered with an athletic supervisor present, may now be taken remotely, but there is no substantial evidence to date indicating that remote administration produces different results or is less effective than supervised testing. This raises the question of whether it is feasible to conduct neurocognitive baseline assessments without direct supervision. This study aims to explore the implications of remote neurocognitive baseline testing during the COVID-19 pandemic, focusing on both the physical location and the level of supervision, to further understand how these factors might affect outcomes.

Hypothesis

The hypothesis is that there will be no statistically significant difference in neurocognitive baseline ImPACT scores between athletes who took the test in a supervised university clinic setting and those who completed it in an unsupervised remote location.

Materials and Methods

Participant sample

The study included 533 participants selected from an anonymized sports medicine database. These participants were NCAA Division II student-athletes (N=533, with 39.2% males and 60.8% females) aged between 17 and 28 years (M=19.80, SD=1.723) (Table 1). The ImPACT assessments were conducted in two separate settings:

• At the University Sports Medicine Clinic (SMC) under the supervision of athletic staff (n=214, 40.2%), and
• At remote locations without supervision (n=319, 59.8%).

These testing sessions were carried out in two distinct academic years based on the study’s structure. The supervised group completed their assessments in the 2021–2022 academic year, while the unsupervised group underwent testing during the 2020–2021 academic year, which coincided with the COVID-19 lockdown period.

The assessments were delivered in the athletes’ primary language, with the majority of participants (n=491, 92.1%) using English. A review of the data revealed that 68 participants had a history of Sport-Related Concussions (SRC). However, all of these athletes had fully recovered and returned to their baseline before participating in the study. Every ImPACT baseline assessment included in this study was confirmed as valid after completion (Table 1).

Table 1. Frequency (n) and valid percent (%) of participant’s sex and testing environment.

Measure impact

The ImPACT is a computerized neurocognitive screening tool that may be administered to those ranging in age from 12–80 years 11 months. The most recent version (ImPACT Version 4) takes approximately 20 minutes to complete and is delivered via a web-based application on a desktop or laptop computer [11].

For remote administration, the user is provided a one-use test code that allows them to complete the test at their convenience. The remote ImPACT is given with instructions to find a quiet testing environment with minimal distractions and may be completed without supervision [12].

The ImPACT was designed to be an objective measure of cognitive abilities such as memory, visual and verbal problem solving, and attention span. Both in its conception and current use, it is intended mainly for utilization with athletes to assess concussions and their neurocognitive effects [13]. ImPACT has two primary uses that consist of baseline testing and post-concussion assessment [14]. Both of these administration procedures can be helpful in measuring deficits resulting from injury, informing treatment planning, and return to learn/play decisions [15].

Test structure

The ImPACT is comprised of various subtests that make up four main composite scores: Verbal memory, visual memory, reaction time, and visual motor speed (at times called processing speed), there are also two supplemental scores, impulse control and total symptom score. There is a post-concussion symptom inventory given at the beginning of the test before any of the cognitive subtests. Also built into the ImPACT are validity indices, with specific cut-off points determined by where 5% of all test-takers fell below the sum of all indicators.

Statistical analysis

Initial analyses were performed to assess whether the assumption of equal variances was satisfied for the samples included in this study. All variables were examined for potential skewness and kurtosis. Reaction time showed a non-normal distribution (leptokurtic) within the remote location group, while total symptom score was non-normally distributed (leptokurtic) across both groups. To evaluate differences between testing environments and supervisory conditions, four univariate ANCOVAs and two Quade ANCOVAs were conducted, incorporating all ImPACT composite and supplemental scores. Age, sex, and education were included as covariates in the analyses. The alpha level was set at 0.05 in advance.

The four univariate ANCOVAs for verbal memory (p=0.787), visual memory (p=0.917), visual motor speed (p=0.261), and impulse control (p=0.783) were not significantly different between groups.

Additionally, the two Quade ANCOVAs for reaction time (p=0.815) and total symptom score (p=0.886) were also not significantly different (Table 2).

Table 2. Mean scores of univariate analysis for each ImPACT composite and supplemental score.

Discussion

This research aimed to explore the effects of environment and supervisory presence on the administration of the ImPACT test among NCAA division II student-athletes. The study was inspired by the widespread shifts in healthcare delivery following the COVID-19 pandemic, which necessitated a transition from in-person to remote-based models. As society adapts to a new normal, it is essential to assess how these rapid changes have positively influenced patient care and assessment practices. In the context of collegiate athletes, remote assessments offer the potential to reduce time constraints and ease the burden on athletes, who often experience fatigue due to their numerous responsibilities, including academics, training, and competition. Allowing athletes to complete cognitive screenings like ImPACT from home could help mitigate the stress associated with their demanding schedules.

To investigate these aspects, the study evaluated whether cognitive assessments could be effectively conducted remotely and whether the testing environment influenced athlete performance. The analysis compared ImPACT composite and supplemental scores between athletes tested in a supervised group setting at a university Sports Medicine Clinic (SMC) and those tested individually in remote, unsupervised settings. At the SMC, testing occurred under the supervision of athletic staff, whereas the remote group completed their assessments independently. The findings supported the hypothesis that neither the testing environment nor supervisory status significantly affected ImPACT performance. These results indicate that collegiate athletes can reliably complete ImPACT assessments remotely and without supervision, without compromising their performance.

The study also addressed the question: “Does the environment affect ImPACT performance?” Previous research, such as that by Netzel and colleagues, investigated the reliability of ImPACT test results when administered to NCAA division I athletes in controlled laboratory settings versus uncontrolled remote locations. Their results aligned with this study, confirming that ImPACT can be reliably administered in remote settings for collegiate athletes.

In addition to environmental factors, this research examined the influence of supervision during ImPACT testing. Results showed no significant differences in performance on composite and supplemental scores between supervised and unsupervised athletes. These findings are consistent with earlier studies indicating that supervision does not significantly affect verbal memory, visual memory, or impulse control scores. However, research by Kuhn and Solomon reported lower visual motor speed and reaction time scores among athletes who completed the test unsupervised compared to those supervised by a physician. This variation may be due to a lack of understanding of speed-based tasks among unsupervised participants. Notably, Kuhn and Solomon’s research focused on high school athletes, emphasizing the need for clear guidance when testing younger individuals. The overall findings of this study support the use of remote, unsupervised administration of ImPACT baseline assessments for collegiate athletes, providing a practical and effective approach to cognitive testing in this population.

Limitations

Some limitations pervaded the present study. It is important to note that the present study only assessed baseline cognitive functioning using ImPACT, so these findings should not be generalized to other cognitive assessments but rather provide direction for future research on the remote administration of assessments.

Because all participants were collegiate athletes identified by the university sports medicine clinic, a selection bias was present as we had a limited ability to gain access to a range of different patients, especially those who were not student-athletes and outside the college age range. This limits the generalizability of our findings to populations beyond student-athletes and outside of our age range. Further, the remote location was not controlled. In other words, the athletes were not supervised, and an athlete could have taken ImPACT in various environments. The assessment was intended to be taken at home, but we were unable to ensure this was done, limiting the external validity of the present study. Most of the participants' primary languages were English, limiting the generalizability of this study to the English administration of ImPACT. To enhance the cultural sensitivity and awareness of the present study, future studies should prioritize the recruitment of participants that do not consider English to be their primary language.

Conclusion

This study is the first of its kind to examine the difference in neurocognitive baseline performance for division II collegiate athletes when administering the ImPACT remotely versus through traditional in- person testing. There were no significant differences found between the athletes who completed their baseline ImPACT at home versus in-person. These findings suggest the ImPACT as a baseline screening measure for college athletes may be completed remotely, to save time and resources for both college athletes and athletic team personnel. Future research should continue to examine the effectiveness of completing remote neurocognitive baseline testing for both youth and collegiate populations.

Declarations

Ethics approval and consent to participate the institutional review board of Nova Southeastern University deemed the study exempt.

Consent for Publication

Consent was waived through the review process as the data is identified and retrospective.

Conflict of Interest

All of the authors declare that they have no conflicts of interest to disclose.

Funding

The authors have no financial or proprietary interests in any material discussed in this article. No funds, grants, or other support was received to assist with the preparation of this manuscript.

Availability of Data

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

Ethics Approval

This research study was conducted retrospectively from data obtained for clinical purposes. We consulted extensively with the IRB of Nova Southeastern University who determined that our study did not need ethical approval. An IRB official waiver of ethical approval was granted from the IRB of Nova Southeastern University.

Author Contribution

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Ryan Bennett, Kayley Slicer, Claire Stafford, and Maya Cohen. The first draft of the manuscript was written by Ryan Bennett, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

Not applicable.

References

1. Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, et al. American medical society for sports medicine position statement: Concussion in sport. Br J Sports Med. 2013;47(1):15-26.

   [Crossref] [Google Scholar] [PubMed]

2. McCrory P, Meeuwisse W, Dvorak J, Aubry M, Bailes J, et al. Consensus statement on concussion in sport-the 5th International Conference on Concussion in Sport held in Berlin, October 2016. Br J Sports Med. 2017;51(11):838-847.

   [Crossref] [Google Scholar] [PubMed]

3. Covassin T, Elbin RJ, Stiller-Ostrowski JL, Kontos AP. Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) practices of sports medicine professionals. J Athl Train. 2009;44(6):639-644.

   [Crossref] [Google Scholar] [PubMed]

4. Lovell MR. What is ImPACT? Information for test takers, parents and guardians. ImPACT Applications. 2022.
5. Moser RS, Schatz P, Neidzwski K, Ott SD. Group versus individual administration affects baseline neurocognitive test performance. Am J Sports Med. 2011;39(11):2325-2330.

   [Crossref] [Google Scholar] [PubMed]

6. French J, Huber P, McShane J, Hollan CL, Elbin RJ, et al. Influence of Test Environment, Age, Sex, and Sport on Baseline Computerized Neurocognitive Test Performance. The Am J Sports Med. 2019;47(13):3263-3269.

   [Crossref] [Google Scholar] [PubMed]

7. Vaughan CG, Gerst EH, Sady MD, Newman JB, Gioia GA. The relation between testing environment and baseline performance in child and adolescent concussion assessment. Am J Sports Med. 2014;42(7):1716-1723.

   [Crossref] [Google Scholar] [PubMed]

8. Lichtenstein JD, Moser RS, Schatz P. Age and test setting affect the prevalence of invalid baseline scores on neurocognitive tests. Am J Sports Med. 2014;42(2):479-484.

   [Crossref] [Google Scholar] [PubMed]

9. Denny KG, Steiner H. External and internal factors influencing happiness in elite collegiate athletes. Child Psychiatry Hum Dev. 2009;40:55-72.

   [Crossref] [Google Scholar] [PubMed]

10. Weber SR, Winkelmann ZK, Monsma EV, Arent SM, Torres-McGehee TM. An examination of depression, anxiety, and self-esteem in collegiate student-athletes. Int J Environ Res Public Health. 2023;20(2):1211.

    [Crossref] [Google Scholar] [PubMed]

11. Lovell MR. Administration and Interpretation Manual Version 4. ImPACT Applications. 2022.
12. Lovell MR. ImPACT At-Home Testing. ImPACT Applications. 2020.
13. Blum K. Embodiment. Scholarship Pressures, and Eating Disordered Behaviors in Division II Collegiate Athletes. West Chester University Doctoral Projects. 2023:171.
14. Netzel L, Moran R, Hopfe D, Salvatore AP, Brown W, et al. Test–retest reliability of remote ImPACT administration. Arch Clin Neuropsychol. 2022;37(2):449-456.

    [Crossref] [Google Scholar] [PubMed]

15. Kuhn AW, Solomon GS. Supervision and computerized neurocognitive baseline test performance in high school athletes: An initial investigation. J Athl Train. 2014;49(6):800-805.

    [Crossref] [Google Scholar] [PubMed]