The OECD PISA tests of 15 year olds are used to rank participating economies and identify educational policy and schooling practice factors that contribute to rank performance in reading, science, and mathematics. This talk will use a confirmatory factor analysis invariance study of the 2009 PISA reading test to identify the limited comparability of performance across participating jurisdictions. Language similarity, reading script, pedagogical priorities, and socio-economic status are used to shed light on the nature of comparability. Suggestions as to how PISA results could be legitimately compared will be made introduced.This talk is based on a published study: Asil, M., & Brown, G. T. L. (2016). Comparing OECD PISA reading in English to other languages: Identifying potential sources of non-invariance. International Journal of Testing, 16(1), 71–93. doi: 10.1080/15305058.2015.1064431