Programme

Have you been exploring the latest AI software?  Perhaps even generating AI poetry on ChatGPT? AI has brought us to a watershed moment in society, which inevitably has implications for assessment. In this plenary we’ll explore the promise and peril of technology in language assessment, pausing to consider opportunities and dilemmas around auto-scoring, adaptive learning, content generation and ethics.  You will (hopefully) emerge with a deeper understanding of the complexities which face assessment organisations, learners and test takers, and apply those insights to next-gen assessment products which leverage new tech while keeping the human at the centre of learning and assessment.

The aim of this plenary is to discuss the role of assessment in the Brazilian education context and its (potential) washback effects. We will take into consideration the implementation of the National Common Curricular Guidelines in 2018, an official document that rules Brazilian elementary and secondary education which established English as a compulsory subject to be taught from the 6th grade onwards. The discussion will be supported by a panorama of Brazilian and foreign large-scale exams applied in the country. In addition, language assessment policies currently adopted by the Brazilian government will be analysed. 

For learning programmes to function efficiently and by inference if they are to be successfully reformed or updated, they should be seen as a system. Within the system there are three core elements: curriculum, delivery, assessment. As with any other system, these elements must be based on a single philosophy of learning supported by clearly defined model of language ability and progression and underpinned by a measurement model. All of this needs to be done with the learner at the heart of the process. 

With this in mind, I will propose that the system, and each of its elements, must meet predetermined academic and pedagogic requirements. In other words, they must conform to clearly stated standards of both language and test development. I will argue that the former of these standards should be driven by a localised CEFR, while the latter should be driven by an established test development model, in this case the socio-cognitive model. 

However, this is not likely to be enough. For a new, reformed or updated system to be function as planned by its developers, it needs to be accepted by the major stakeholder groups that make up the population of the context-of-use. In order to ensure that this can happen, developers should develop a clearly stated theory of change (put simply: what they hope to achieve) and a corresponding theory of action (put simply: how they intend to achieve it). It is critical that these are based on the needs and expectations of the key stakeholders, as is the need to communicate with these groups and individuals in a timely and appropriate manner. In this way, the social consequences (both intended and unintended) of the implementation of the system can be closely monitored and effectively dealt with. This thinking is reflected in two approaches to test development and validation: the Comprehensive Learning System (O’Sullivan, 2020) and the Integrated Arguments approach (Chalhoub-Deville and O’Sullivan, 2020). Returning to the concept of placing the learner at the centre of the process, I conclude by reflecting on the increasingly important area of equality, diversity and inclusion, suggesting that this forms the fourth pillar upon which a success system is built.