New Zealand's National Education Monitoring Project 1995-2009. The Pre-Conception, Conception, Realisation, and Contribution of an Educationally Principled Large-Scale Assessment Programme

Abstract

This thesis is a researched account of the factors that gave rise to New Zealand's National Education Monitoring Project, the design and practice of the project, and the contribution it made to knowledge of student achievement in all areas of the national curriculum. The purposes of this Ministry of Education funded project were to meet public accountability and information requirements by identifying and reporting patterns and trends in educational performance, and providing high quality information that policy makers, curriculum planners and educators could use to debate and review educational practices and resourcing. From 1995 to 2009, this large-scale research programme annually assessed nationally representative samples of approximately 2,800 year 4 and year 8 students so that their achievements could be reported and debated. Clearly defined principles drawn from perspectives of both theory and practice underpinned the design and operation of the project at every point. Following detailed investigations into the design and implementation of system monitoring in other educational jurisdictions, it was found that some features were common to most, such as multiple matrix sampling, whereas others, such as the choice of assessment formats and subject coverage, were variously predicated on population size, resourcing, standards-based curricula, and political interests. Performance-based assessment, while extensively debated by measurement specialists, was gaining widespread acceptance for its potential to provide markedly greater strength of validity in the interpretation and use of assessment results. This was particularly evident in contexts where there was a strong desire to emphasise the formative functions of assessment (assessment to improve learning) more so than summative functions (assessment for which the endpoint is reporting). Regardless, traditional paper-and-pencil formats tended to prevail, with comparatively small proportions of performance tasks. Consequently, very few examples were available internationally of rich performance tasks suited to large-scale assessment programmes across the breadth of curriculum that might benefit a New Zealand programme. Other notable differences among monitoring systems were found in the approaches used to administer tasks, and in the statistical and descriptive methods used for analysing and reporting results. Research in the design and practice of monitoring systems elsewhere was used extensively in deliberations that led to the development and implementation of a programme design that was suited to the New Zealand context and purposes. The result, however, was not a programme that mirrored another. The National Education Monitoring Project developed and used processes that were somewhat unique among large-scale programmes. Those processes included the approaches to administering assessment tasks and recording student responses, the extensive use of performance-based task formats, the engagement of practising teachers for administering and scoring the tasks, the full and regularised coverage of all areas of the national curriculum, task-by-task descriptive reporting, and examination of trends in student achievement by maintaining intact tasks from one assessment cycle to the next with re-sampled populations, rather than using statistical linking. In sum, NEMP's contribution to knowledge included a vast amount of information on what New Zealand primary school students know and can do across a wide range of learning outcomes. Added to this, the Project contributed to the field of knowledge on educational assessment perspectives generally, and to large-scale applications in particular.