Programme for International Student Assessment
|Programme for International Student Assessment|
|Purpose/focus||Comparison of education attainment across the world|
|Location||2 rue André Pascal, 75775 Paris Cedex 16|
|Membership||59 government education departments|
|Head of the Early Childhood and Schools Division||Michael Davidson|
|Main organ||PISA Governing Body (Chair – Lorna Bertrand, England)|
The Programme for International Student Assessment (PISA) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) in member and non-member nations of 15-year-old school pupils' scholastic performance on mathematics, science, and reading. It was first performed in 2000 and then repeated every three years. It is done with view to improving education policies and outcomes. The data has increasingly been used both to assess the impact of education quality on incomes and growth and for understanding what causes differences in achievement across nations.
470,000 15-year-old students representing 65 nations and territories participated in PISA 2009. An additional 50,000 students representing nine nations were tested in 2010.
The Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) by the International Association for the Evaluation of Educational Achievement are similar studies.
- 1 Framework
- 2 Development and implementation
- 3 Method of testing
- 4 Results
- 5 Reception
- 6 Research on causes of country differences
- 7 See also
- 8 References
- 9 Further reading
- 10 External links
PISA stands in a tradition of international school studies, undertaken since the late 1950s by the International Association for the Evaluation of Educational Achievement (IEA). Much of PISA's methodology follows the example of the Trends in International Mathematics and Science Study (TIMSS, started in 1995), which in turn was much influenced by the U.S. National Assessment of Educational Progress (NAEP). The reading component of PISA is inspired by the IEA's Progress in International Reading Literacy Study (PIRLS).
The PISA mathematics literacy test asks students to apply their mathematical knowledge to solve problems set in real-world contexts. To solve the problems students must activate a number of mathematical competencies as well as a broad range of mathematical content knowledge. TIMSS, on the other hand, measures more traditional classroom content such as an understanding of fractions and decimals and the relationship between them (curriculum attainment). PISA claims to measure education's application to real-life problems and lifelong learning (workforce knowledge).
In the reading test, "OECD/PISA does not measure the extent to which 15-year-old students are fluent readers or how competent they are at word recognition tasks or spelling." Instead, they should be able to "construct, extend and reflect on the meaning of what they have read across a wide range of continuous and non-continuous texts."
Development and implementation
Developed from 1997, the first PISA assessment was carried out in 2000. The results of each period of assessment take about one year and a half to be analysed. First results were published in November 2001. The release of raw data and the publication of technical report and data handbook only took place in spring 2002. The triennial repeats follow a similar schedule; the process of seeing through a single PISA cycle, start-to-finish, always takes over four years.
Every period of assessment focuses on one of the three competence fields of reading, math, science; but the two others are tested as well. After nine years, a full cycle is completed: after 2000, reading was again the main domain in 2009.
|Period||Main focus||# OECD countries||# other countries||# students||Notes|
|2000||Reading||28||4||265,000||The Netherlands disqualified from data analysis. 11 additional non-OECD countries took the test in 2002|
|2003||Mathematics||30||11||275,000||UK disqualified from data analysis. Also included test in problem solving.|
|2009||Reading||34||33?||Results made available on 7 December 2010.|
PISA is sponsored, governed, and coordinated by the OECD. The test design, implementation, and data analysis is delegated to an international consortium of research and educational institutions led by the Australian Council for Educational Research (ACER). ACER leads in developing and implementing sampling procedures and assisting with monitoring sampling outcomes across these countries. The assessment instruments fundamental to PISA's reading, mathematics, science, problem-solving, computer-based testing, background and contextual questionnaires are similarly constructed and refined by ACER. ACER also develops purpose-built software to assist in sampling and data capture, and analyses all data. The source code of the data analysis software is not made public.
Method of testing
The students tested by PISA are ages between 15 years and 3 months and 16 years and 2 months at the beginning of the assessment period. The school year pupils are in is not taken into consideration. Only students at school are tested, not home-schoolers. In PISA 2006, however, several countries also used a grade-based sample of students. This made it possible to study how age and school year interact.
To fulfill OECD requirements, each country must draw a sample of at least 5,000 students. In small countries like Iceland and Luxembourg, where there are fewer than 5,000 students per year, an entire age cohort is tested. Some countries used much larger samples than required to allow comparisons between regions.
Each student takes a two-hour handwritten test. Part of the test is multiple-choice and part involves fuller answers. There are six and a half hours of assessment material, but each student is not tested on all the parts. Following the cognitive test, participating students spend nearly one more hour answering a questionnaire on their background including learning habits, motivation and family. School directors fill in a questionnaire describing school demographics, funding, etc.
In selected countries, PISA started experimentation with computer adaptive testing.
Countries are allowed to combine PISA with complementary national tests.
Germany does this in a very extensive way: On the day following the international test, students take a national test called PISA-E (E=Ergänzung=complement). Test items of PISA-E are closer to TIMSS than to PISA. While only about 5,000 German students participate in the international and the national test, another 45,000 take only the latter. This large sample is needed to allow an analysis by federal states. Following a clash about the interpretation of 2006 results, the OECD warned Germany that it might withdraw the right to use the "PISA" label for national tests.
From the beginning, PISA has been designed with one particular method of data analysis in mind. Since students work on different test booklets, raw scores must be 'scaled' to allow meaningful comparisons. This scaling is done using the Rasch model of item response theory (IRT). According to IRT, it is not possible to assess the competence of students who solved none or all of the test items. This problem is circumvented by imposing a Gaussian prior probability distribution of competences.
One and the same scale is used to express item difficulties and student competences. The scaling procedure is tuned such that the a posteriori distribution of student competences, with equal weight given to all OECD countries, has mean 500 and standard deviation 100.
The official reports only contain domain-specific scores and do not combine the different domains into an overall score. The final scoring is adjusted so that the OECD average in each domain is 500 and the standard deviation is 100.
||This article contains embedded lists that may be poorly defined, unverified or indiscriminate. (September 2011)|
All PISA results are broken down by countries. Public attention concentrates on just one outcome: achievement mean values by countries. These data are regularly published in form of "league tables".
The following table gives the mean achievements of OECD member countries in the principal testing domain of each period:
In the official reports, country rankings are communicated in a more elaborate form: not as lists, but as cross tables, indicating for each pair of countries whether or not mean score differences are statistically significant (unlikely to be due to random fluctuations in student sampling or in item functioning). In favorable cases, a difference of 9 points is sufficient to be considered significant.
In some popular media, test results from all three literacy domains have been consolidated in an overall country ranking. Such meta-analysis is not endorsed by the OECD. The official reports only contain domain-specific country scores. In part of the official reports, however, scores from a period's principal testing domain are used as proxy for overall student ability.
Top results for the main areas of investigation of PISA, in 2000, 2003 and 2006:
Top 10 countries for Pisa 2006 results in Mathematics, Science and Reading:
|Programme for International Student Assessment (2006)
(OECD member countries in boldface)
The PISA 2009 results in Maths, Science and Reading for all 34 OECD members and 37 partner countries. Of the partner countries, only selected areas of three countries—India, Venezuela and China—were assessed. Due to scheduling constraints, 10 of those partners actually carried out their tests in 2010, not 2009.
The results for 2012 was presented on 3 December 2013. The test had a special focus on mathematics. About 500 000 pupils in 65 countries and local administrations (Chinese cities) participated. The mean score in mathematics was, with Shanghai scoring the best result with 613, followed by other East Asian areas. Liechtenstein, Switzerland and the Netherlands scored best among the European nations. The results for USA and the United Kingdom were in the middle and showed no great change from previous tests. Sweden was the country which had the greatest fall in their performance in mathematics over the last ten-year, with a similar falling trend also in the two other subjects and leading politicians in Sweden expressed great worry over the results.
The results showed that boys on average scored better than girls in mathematics, girls scored better than boys in reading while the score for the two genders were quite similar in science.
The test found that "21% of the variation in countries' mean scores can be predicted on the basis of their per capita GDP", while 30% of variation between countries correlated to spending on education.
|Programme for International Student Assessment (2012)
(OECD members as of the time of the study in boldface)
Comparison with other studies
The correlation between PISA 2003 and TIMSS 2003 grade 8 country means is 0.84 in mathematics, 0.95 in science. The values go down to 0.66 and 0.79 if the two worst performing developing countries are excluded. Correlations between different scales and studies are around 0.80. The high correlations between different scales and studies indicate common causes of country differences (e.g. educational quality, culture, wealth or genes) or a homogenous underlying factor of cognitive competence. European Economic Area countries perform slightly better in PISA; the Commonwealth of Independent States and Asian countries in TIMSS. Content balance and years of schooling explain most of the variation.
For many countries, the results from PISA 2000 were surprising. In Germany and the United States, for example, the comparatively low scores brought on heated debate about how the school system should be changed. Some headlines in national newspapers, for example, were:
- "La France, élève moyen de la classe OCDE" (France, average student of the OECD class) Le Monde, 5 December 2001
- "Miserable Noten für deutsche Schüler" (Abysmal marks for German students) Frankfurter Allgemeine Zeitung, 4 December 2001
- "Are we not such dunces after all?" The Times, United Kingdom, 6 December 2001
- "Economic Time Bomb: U.S. Teens Are Among Worst at Math" The Wall Street Journal 7 December 2004
- "Preocupe-se. Seu filho é mal educado." (Be worried. Your child is badly educated.) Brazil. Veja, 7 November 2007
- "La educación española retrocede" (Spanish education moving backwards) El País 5 December 2007
- "Finnish teens score high marks in latest PISA study" Helsingin Sanomat 30 November 2007
Education professor Yong Zhao has noted the PISA 2009 did not receive much attention in the Chinese media, and that the high scores in China are due to excessive workload and testing, adding that it's "no news that the Chinese education system is excellent in preparing outstanding test takers, just like other education systems within the Confucian cultural circle: Singapore, Korea, Japan, and Hong Kong."
Of the 74 countries tested in the PISA 2009 cycle including the "+" nations, the two Indian states came up 72nd and 73rd out of 74 in both reading and maths, and 73rd and 74th in science. India's poor performance may not be linguistic as some suggested. 12.87% of US students, for example, indicated that the language of the test differed from the language spoken at home. while 30.77% of Himachal Pradesh students indicated that the language of the test differed from the language spoken at home, a significantly higher percent However, unlike American students, those Indian students with a different language at home did better on the PISA test than those with the same language. India's poor performance on the PISA test is consistent with India's poor performance in the only other instance when India's government allowed an international organization to test its students and consistent with India's own testing of its elite students in a study titled Student Learning in the Metro 2006. These studies were conducted using TIMSS questions. The poor result in PISA was greeted with dismay in the Indian media. The BBC reported that as of 2008, only 15% of India's students reach high school.
India pulled out of the 2012 round of PISA testing, in August 2012, with the Indian government attributing its action to the unfairness of PISA testing to Indian students. The Indian Express reported on 9/3/2012 that "The ministry (of education) has concluded that there was a socio-cultural disconnect between the questions and Indian students. The ministry will write to the OECD and drive home the need to factor in India's "socio-cultural milieu". India's participation in the next PISA cycle will hinge on this". The Indian Express also noted that "Considering that over 70 nations participate in PISA, it is uncertain whether an exception would be made for India".
In June 2013, the Indian government, still concerned with the future prospect of fairness of PISA testing relating to Indian students, again pulled India out from the 2015 round of PISA testing.
It quotes University of Bristol Professor Harvey Goldstein, who explains that when the OECD tries to rule out questions suspected of bias, it can have the effect of "smoothing out" key differences between countries. "That is leaving out many of the important things,” he warns. "They simply don't get commented on. What you are looking at is something that happens to be common. But (is it) worth looking at? PISA results are taken at face value as providing some sort of common standard across countries. But as soon as you begin to unpick it, I think that all falls apart."
The article cites University of Copenhagen Professor Svend Kreiner, who looked at the reading results for 2006 in detail. He says that only about 10 per cent of the students who took part in PISA were tested on all 28 reading questions. "This in itself is ridiculous,” Kreiner told the TES. "Most people don't know that half of the students taking part in PISA (2006) do not respond to any reading item at all. Despite that, PISA assigns reading scores to these children."
Queen's University Belfast mathematician Dr. Hugh Morrison argues that the statistical model underlying PISA has a fundamental, insoluble mathematical error that renders Pisa rankings "valueless". Goldstein told the TES that Morrison highlights “an important technical issue” rather than the “profound conceptual error”. PISA “has been used inappropriately and some of the blame for that lies with PISA itself. I think it tends to say too much for what it can do and it tends not to publicise the negative or the weaker aspects.”
Research on causes of country differences
Large international student assessment programs such as PISA and TIMSS have provided essential data for many recent analyses of how student achievement affects society at large, such as economic development, democratization and health.
Although PISA and TIMSS officials and researchers themselves generally refrain from hypothesizing about the large and stable differences in student achievement between countries, other researchers have investigated single educational factors like central exams private schools or streaming between schools at later age. An extensive literature related to cross-countries difference in scores has also developed since 2000.
An evaluation of the 2003 results showed that countries that spent more on education did not necessarily do better. Australia, Belgium, Canada, the Czech Republic, Finland, Japan, South Korea, New Zealand and the Netherlands spent less but did relatively well, whereas the United States spent much more but was below the OECD average. The Czech Republic, in the top ten, spent only one third as much per student as the United States did, for example, but the USA came 24th out of 29 countries compared.
The stable, high marks of Finnish students have attracted a lot of attention. According to Hannu Simola the results are due to a paradoxical mix of progressive policies implemented through a rather conservative pedagogic setting, where the high levels of teachers' academic preparation, social status, professionalism and motivation for the job are concomitant with the adherence to traditional roles and methods by both teachers and pupils in Finland's changing, but still rather authoritarian culture. Others have suggested that Finland's low poverty rate is a reason for its success.
Finnish education reformer Pasi Sahlberg suggests that the reason for Finland's high educational achievements is because of the country's focus on access to quality education for all, as opposed to a focus on competition among teachers and schools. Lynn and Meisenberg (2010) found very high correlations (r>0.90) between mean student assessment results from PISA, TIMSS, PIRLS and others and IQ measurements at the country data level.
The Finnish language may be one reason for the good results of Finland.
Another point made in the evaluation was that students with higher-earning parents are better-educated and tend to achieve higher results. This was true in all the countries tested, although more obvious in certain countries, such as Germany.
In 2012, 2010, the 2009 Program for International Student Assessment (PISA) results revealed that Shanghai students scored the highest in the world in every category (Mathematics, Reading and Science). The OECD described Shanghai as a pioneer of educational reform, noting that "there has been a sea change in pedagogy". OECD point out that they "abandoned their focus on educating a small elite, and instead worked to construct a more inclusive system. They also significantly increased teacher pay and training, reducing the emphasis on rote learning and focusing classroom activities on problem solving."
OECD has also noted that even in rural China results approached average levels for the OECD countries: "Citing further, as-yet unpublished OECD research, Mr Schleicher said, 'We have actually done Pisa in 12 of the provinces in China. Even in some of the very poor areas you get performance close to the OECD average.'" For a developing country, China's 99.4% enrolment in primary education is already, as the OECD puts it, "the envy of many countries" while junior secondary school participation rates in China are now 99%. But in Shanghai not only has senior secondary school enrolment attained 98% but admissions into higher education have achieved 80% of the relevant age group. That this growth reflects quality, not just quantity, is confirmed clearly by the OECD's ranking of Shanghai's secondary education as world number one. According to the OECD, China has also expanded school access, and moved away from learning by rote. "'The last point is key: Russia performs well in rote-based assessments, but not in Pisa,' says Schleicher, head of the indicators and analysis division at the OECD’s directorate for education. 'China does well in both rote-based and broader assessments.'"
Two studies have compared high achievers in mathematics on the PISA and the U.S. National Assessment of Educational Progress (NAEP). Comparisons were made between those scoring at the "advanced" and "proficient" levels in mathematics on the NAEP with the corresponding performance on the PISA. Overall, 30 nations had higher percentages than the U.S. of students at the "advanced" level of mathematics. The only OECD countries with worse results were Portugal, Greece, Turkey, and Mexico. Six percent of U.S. students were "advanced" in mathematics compared to 28 percent in Taiwan. The highest ranked state in the U.S. (Massachusetts) was just 15th in the world if it was compared with the nations participating in the PISA. 31 nations had higher percentages of "proficient" students than the U.S. Massachusetts was again the best U.S. state, but it ranked just ninth in the world if compared with the nations participating in the PISA.
Comparisons with results for the Trends in International Mathematics and Science Study (TIMSS) appear to give different results—suggesting that the U.S. states actually do better in world rankings. The difference in apparent rankings is, however, almost entirely accounted for by the sampling of countries. PISA includes all of the OECD countries, while TIMSS is much more weighted in its sampling toward developing countries.
University of Southern California professor Stephen Krashen and Mel Riddile of the NASSP say that low performance in the United States is closely related to American poverty, but the same reasoning applies to other countries. US schools with poverty rates comparable to Finland's (below 10%) outperform Finland and schools in the 10–24% range aren't far behind.
Reduced school lunch participation is the only available intra-poverty indicator for US schoolchildren; areas with less than 10% of the students having free or reduced price lunch averaged 551 (higher than any other OECD country). In comparison with the rest other OECD countries (which have tabled figures on children living in relative poverty):
|Country||Percent of reduced school lunches (US)
Percent of relative child poverty (Other OECD countries)
|United States||< 10%||551|
|United States||> 75%||446|
- Teaching And Learning International Survey (TALIS)
- Trends in International Mathematics and Science Study (TIMSS)
- Gender gaps in mathematics and reading in PISA 2009
- Hanushek, Eric A., and Ludger Woessmann. 2011. "The economics of international differences in educational achievement." In Handbook of the Economics of Education, Vol. 3, edited by Eric A. Hanushek, Stephen Machin, and Ludger Woessmann. Amsterdam: North Holland: 89–200.
- PISA 2009 Technical Report, 2012, OECD, http://www.oecd.org/dataoecd/60/31/50036771.pdf
- Hefling, Kimberly. "Asian nations dominate international test". Yahoo!.
- "Chapter 2 of the publication 'PISA 2003 Assessment Framework'" (pdf). Pisa.oecd.org.
- C. Füller: Pisa hat einen kleinen, fröhlichen Bruder. taz, 5.12.2007 
- The scaling procedure is described in nearly identical terms in the Technical Reports of PISA 2000, 2003, 2006. It is similar to procedures employed in NAEP and TIMSS. According to J. Wuttke Die Insignifikanz signifikanter Unterschiede. (2007, in German), the description in the Technical Reports is incomplete and plagued by notational errors.
- PISA 2009. http://www.pisa.oecd.org/document/61/0,3746,en_32252351_32235731_46567613_1_1_1_1,00.html
- OECD (2001) p. 53; OECD (2004a) p. 92; OECD (2007) p. 56.
- E.g. OECD (2001), chapters 7 and 8: Influence of school organization and socio-economic background upon performance in the reading test. Reading was the main domain of PISA 2000.
- Multi-dimensional Data Request, OECD, 2010, retrieved 2012-06-28
- PISA 2009 Results: Executive Summary (Figure 1 only), OECD, 2010, retrieved 2012-06-28
- Walker, Maurice (2011), PISA 2009 Plus Results, OECD, retrieved 2012-06-28
- Stephanie Simon (3 December 2013) PISA results show ‘educational stagnation’ in U.S. Politico. Retrieved 3 December 2013
- Richard Adams (3 December 2013)Swedish results fall abruptly as free school revolution falters The Guardian. Retrieved 3 December 2013
- Jens Kärrman (3 Decembere 2013) Löfven om Pisa: Nationell kris Dagens Nyheter. Retrieved 3 December 2013 (Swedish)
- Datablog Pisa 2012 results: which country does best at reading, maths and science? The Guardian. Retrieved 3 December 2013
- PISA 2012 Results in Focus, OECD, 3 December 2013, retrieved 4 December 2013
- M. L. Wu: A Comparison of PISA and TIMSS 2003 achievement results in Mathematics. Paper presented at the AERA Annual Meeting, New York, March 2008. .
- Preocupe-se. Seu filho é mal educado Veja, 7 November 2007. Retrieved 13 April 2013 (Breton)
- "Waiting for "Superman" trailer". Retrieved 8 October 2010.
- Yong Zhao (10 December 2010), A True Wake-up Call for Arne Duncan: The Real Reason Behind Chinese Students Top PISA Performance
- "Database – PISA 2009". Pisa2009.acer.edu.au.
- Vishnoi, Anubhuti (7 January 2012), "Poor PISA ranks: HRD seeks reason", The Indian Express
- Masani, Zareer (27 February 2008). "India still Asia's reluctant tiger". BBC News.
- Hemali Chhapia, TNN (3 August 2012). "India backs out of global education test for 15-year-olds". The Times of India.
- "Poor PISA score: Govt blames ‘disconnect’ with India". The Indian Express. 3 September 2012.
- "India chickens out of international students assessment programme again". The Times of India. 1 June 2013.
- "Is Pisa fundamentally flawed? – news". TES.
- Hanushek, Eric; Woessmann, Ludger (2008), "The role of cognitive skills in economic development", Journal of Economic Literature 46 (3): 607–668, doi:10.1257/jel.46.3.607
- Rindermann, Heiner; Ceci, Stephen J (2009), "Educational policy and country outcomes in international cognitive competence studies", Perspectives on Psychological Science 4 (6): 551–577, doi:10.1111/j.1745-6924.2009.01165.x
- Bishop, John H (1997), "The effect of national standards and curriculum-based exams on achievement", American Economic Review 87 (2): 260–264
- Hanushek, Eric; Woessmann, Ludger (2006), "Does educational tracking affect performance and inequality? Differences-in-differences evidence across countries", Economic Journal 116 (510): C63–C76
- Simola, H. (2005). The Finnish miracle of PISA: Historical and sociological remarks on teaching and teacher education. Comparative Education, 41, 455–470.
- "The Economics Behind International Education Rankings" National Educational Association
- Riddile, Mel (15 December 2010), PISA: It's Poverty Not Stupid, National Association of Secondary School Principals
- Cleland, Elizabeth. "What Americans Keep Ignoring About Finland's School Success – Anu Partanen". The Atlantic.
- Lynn, R. & Meisenberg, G. (2010). National IQs calculated and validated for 108 nations. Intelligence, 38, 353–360.
- (2013 version) Why does Finnish give better PISA results?
- Gumbel, Peter (7 December 2010), "China Beats Out Finland for Top Marks in Education", Time, retrieved 27 June 2012
- Cook, Chris (7 December 2010), "Shanghai tops global state school rankings", Financial Times, retrieved 28 June 2012
- Mance, Henry (7 December 2010), "Why are Chinese schoolkids so good?", Financial Times, retrieved 28 June 2012
- Paul E. Peterson, Ludger Woessmann, Eric A. Hanushek, and Carlos X. Lastra-Anadón (2011) "Are U.S. students ready to compete? The latest on each state's international standing." Education Next 11, no. 4 (Fall): 51–59. http://educationnext.org/are-u-s-students-ready-to-compete/
- Eric A. Hanushek, Paul E. Peterson, and Ludger Woessmann (2011) "Teaching math to the talented." Education Next 11, no. 1 (Winter): 10–18. http://educationnext.org/teaching-math-to-the-talented/
- Gary W. Phillips (2007) Chance favors the prepared mind: Mathematics and science indicators for comparing states. Washington: American Institutes for Research (14 November); Gary W. Phillips (2009) The Second Derivative:International Benchmarks in Mathematics For U.S. States and School Districts. Washington, DC: American Institutes for Research (June).
- "How poverty affected U.S. PISA scores" The Washington Post
- "Stratifying PISA scores by poverty rates suggests imitating Finland is not necessarily the way to go for US schools". Simply Statistics. 23 August 2013.
- "Child poverty statistics: how the UK compares to other countries", The Guardian. The same UNICEF figures were used by Riddile.
- Highlights From PISA 2009, Table 3.
Official websites and reports
- OECD/PISA website
- OECD (1999): Measuring Student Knowledge and Skills. A New Framework for Assessment. Paris: OECD, ISBN 92-64-17053-7 
- OECD (2001): Knowledge and Skills for Life. First Results from the OECD Programme for International Student Assessment (PISA) 2000.
- OECD (2003a): The PISA 2003 Assessment Framework. Mathematics, Reading, Science and Problem Solving Knowledge and Skills. Paris: OECD, ISBN 978-92-64-10172-2 
- OECD (2004a): Learning for Tomorrow's World. First Results from PISA 2003. Paris: OECD, ISBN 978-92-64-00724-6 
- OECD (2004b): Problem Solving for Tomorrow's World. First Measures of Cross-Curricular Competencies from PISA 2003. Paris: OECD, ISBN 978-92-64-00642-3
- OECD (2005): PISA 2003 Technical Report. Paris: OECD, ISBN 978-92-64-01053-6
- OECD (2007): Science Competencies for Tomorrow's World: Results from PISA 2006 
Reception and political consequences
- A. P. Jakobi, K. Martens: Diffusion durch internationale Organisationen: Die Bildungspolitik der OECD. In: K. Holzinger, H. Jörgens, C. Knill: Transfer, Diffusion und Konvergenz von Politiken. VS Verlag für Sozialwissenschaften, 2007.
- N. Mons, X. Pons: The reception and use of Pisa in France.
- E. Bulmahn [then federal secretary of education]: PISA: the consequences for Germany. OECD observer, no. 231/232, May 2002. pp. 33–34.
- H. Ertl: Educational Standards and the Changing Discourse on Education: The Reception and Consequences of the PISA Study in Germany. Oxford Review of Education, v 32 n 5 pp 619–634 Nov 2006.
- S. Grek, M. Lawn, J. Ozga: Study on the Use and Circulation of PISA in Scotland. 
- S. Hopmann, G. Brinek, M. Retzl (eds.): PISA zufolge PISA. PISA According to PISA. LIT-Verlag, Wien 2007, ISBN 3-8258-0946-3 (partly in German, partly in English)
- T. Jahnke, W. Meyerhöfer (eds.): PISA & Co – Kritik eines Programms. Franzbecker, Hildesheim 2007 (2nd edn.), ISBN 978-3-88120-464-4 (in German)
- R. Münch: Globale Eliten, lokale Autoritäten: Bildung und Wissenschaft unter dem Regime von PISA, McKinsey & Co. Frankfurt am Main : Suhrkamp, 2009. ISBN 978-3-518-12560-1 (in German)
- J . Wuttke: Critical online bibliography
- Correlation of 2009-PISA Scores with National GDP PISA versus GDP