Case studies have revealed neurological problems in severely affected COVID-19 patients. However, there is little information regarding the nature and broader prevalence of cognitive problems post-infection or across the full spread of severity. We analysed cognitive test data from 84,285 Great British Intelligence Test participants who completed a questionnaire regarding suspected and biologically confirmed COVID-19 infection. People who had recovered, including those no longer reporting symptoms, exhibited significant cognitive deficits when controlling for age, gender, education level, income, racial-ethnic group and pre-existing medical disorders. They were of substantial effect size for people who had been hospitalised, but also for mild but biologically confirmed cases who reported no breathing difficulty. Finer grained analyses of performance support the hypothesis that COVID-19 has a multi-system impact on human cognition.