The Ai Group Centre for Education and Training was invited to contribute to doctoral research work by Dr Roberto Schurch on the impact of the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC) on Australia’s adult education policies. A copy of the thesis produced by Dr Schurch is available here.
The following is the abstract from the thesis, which has been approved for reproduction by the author:
This study is a comparative project about a comparative project: the Programme for the International Assessment of Adult Competencies (PIAAC) and its ‘Survey of Adult Skills’ developed by the Organisation for Economic Co-operation and Development (OECD). PIAAC is an International Large-Scale Assessment (ILSA) that assesses functional literacy (reading), numeracy, and problem-solving competencies in adults aged 16 to 65 years, comparing results between and within 42 countries which have participated in the program (OECD, 2016b, 2022b).
The research aims to compare how PIAAC has intervened in policy governance in education at the supranational level and within the national contexts of Australia, Chile, and Singapore. These countries present an excellent opportunity to explore the similarity and differences between PIAAC’s productivity in varied cultural, institutional, economic, and social settings, including countries with diverse levels of involvement with the OECD and results in PIAAC. This research is informed by a broad Critical Policy Sociology (CPS) approach and, following Ball’s ‘theoretical toolbox’ (2005, p. 43), it integrates multiple theories and conceptualisations to understand the implications of ILSAs in education. Rather than analysing PIAAC as an objective and neutral tool, this research draws upon Gorur’s conceptualisation of ILSAs as complex socio-technical assemblages (2017).
The study proposes a pluri-scalar and multi-dimensional model to analyse the role of PIAAC in education policy governance. In particular, the study focuses on the actors and relationships involved in this assemblage, the rationales, and preconditions to participate in PIAAC, the contextualisation of its standards, and the uses and implications of the findings of PIAAC (described as the ‘ARCU model’). The study employed Bartlett and Vavrus’s (2017) Comparative Case Study (CCS) approach as the research design to elaborate on the specificity of the ARCU model as a comparative endeavour. Thus, following CCS, the study draws upon a vertical analysis across levels for each case and horizontal analyses across cases. In this research, the three countries are not considered as isolated units of analysis but as key nodes of each of the three vertical cases.
The methods combined document analysis, investigative research on the internet, and interviews with users and implementers of PIAAC at different scales bringing together voices from inside this ‘international testing machine’ (Addey, 2019, p. 1). These perspectives included my own experience as a former officer involved in the national implementation of PIAAC in Chile and that of officers and policymakers from the OECD, specialists from PIAAC expert groups, and public servants, policymakers, and researchers directly involved with PIAAC in the three countries. Altogether, these methods made it possible to problematise the productivity of PIAAC beyond its representation in the media, which has been the dominant approach so far, with findings indicating the limited influence of PIAAC in the analysed countries (Cort & Larson, 2015; Hamilton, 2018a; Yasukawa, 2019; Yasukawa et al., 2017).
Contrary to many previous studies, this research found subtle but powerful interventions of PIAAC in policy texts and discourses, which have been exerted through a process of ‘customisation’. Thus, this study showed that PIAAC had intervened in education policies not so much through its capacity to set common standards but, on the contrary, due to the exceptional flexibility offered to the participating countries to personalise a particular assemblage that fitted their capacities, needs, and interests. As demonstrated, this customisation was deployed through varied configurations of actors put in place in the countries for the national implementation of this ILSA. Moreover, this customisation was exercised through the multiple rationales offered to national actors to stimulate their interest in participating in PIAAC. In addition, PIAAC enabled countries to contextualise the international standards to their technical capacities and policy needs, including selecting alternatives and negotiating, modifying, and even ignoring some of these standards. Lastly, PIAAC allowed multiple types of usage of its results by various actors in several domains under different circumstances. As this research demonstrated, PIAAC results have intervened in educational policies and the productivity sector, via government and non-government actors, through policy discourses and texts, for reform agendas supportive and against standardised testing, under progressive and conservative national governments, and at national and sub-national levels.
In summary, the research argues that such instance of customisation represents an additional ‘soft power’ mechanism deployed by the OECD to gain influence in education policy governance, particularly in Adult Lifelong Learning and related areas. This customisation, however, showed the limited capacity of PIAAC to embrace social justice principles. As this study evidenced, the possibility of customisation of PIAAC was not equitable among the countries analysed, nor between social groups within them. Thus, different forms of ‘otherness’, such as nations from the Global South (e.g., Chile) and groups within and across the analysed countries (e.g., first nations peoples, migrant workers, people living in isolated areas, and those speaking non-dominant languages), had limited opportunities to contribute to the creation of knowledge about adult competencies and make visible their capacities and needs through this ILSA. As the study revealed, rather than a simple imposition from the OECD, the intervention of PIAAC, including the marginalisation and reproduction of cultural and socioeconomic injustices, was exerted jointly with national actors involved in this ILSA. This imbricated intervention of PIAAC manifests the contemporary shape of education governance and some of the adverse implications of quantification under current ‘governance by numbers’ processes.
