How Oxford University Press is engaging with contemporary educational research through the ‘Oxford Impact’ initiative
Every day, educational publishers promote policy change around the world through partnerships with policymakers, schools, teachers and parents. The partnerships that publishers form with educational researchers are of increasing importance in an era that includes What Works Clearinghouse (USA) and What Works Centres (UK).
Increasing demand for evidence and OUP’s response: the Oxford Impact initiative
We are witnessing an ever-increasing global demand for evidence of the effects of learning materials, including from governments, policymakers, educational bodies, quality assurance agencies, and from our customers when they consider a purchase (especially if a purchase involves significant budget spend). The Oxford Impact initiative is OUP’s response to this demand for evidence and is the result of collaboration between an educational publisher (OUP) and educational stakeholders (including educational researchers). Specifically, the initiative is the result of OUP’s collaboration with an education charity (the UK National Foundation for Educational Research, NFER), practitioners, professional bodies and educational researchers. Professor Pamela Sammons, Oxford University Department of Education, acted as a sounding board and reviewer. This moved OUP towards building a decision-making tool used for rigorous evaluations of OUP’s educational resources, services and materials: The Oxford Impact Framework.
What do we mean by ‘impact’?
By ‘impact’ OUP refers to an identified change that one of our educational resources has on the people it is intended to help or benefit. By ‘impact study’ we refer to the research process that OUP engages in to identify whether or not the resource is affecting the change that it labels ‘impact’. We also consider the extent to which a change can be attributed to that particular resource. This is distinct from market research and user testing.
Impact studies span all phases, from pre-primary to adult, and look at a particular beneficiary group – learners, teachers, senior leaders or parents/carers – and a specific change that our material is intended to deliver, captured in the form of an ‘impact objective’. Examples include: ‘increased motivation to learn’, ‘increased confidence’, ‘improved collaboration with a school’, etc. In terms of methodology, these include studies gathering ‘perceptions of impact’ through to Randomized Controlled Trials (RCTs). When we undertake an impact study we collaborate with educators, parents, senior leaders, learners, researchers and policymakers worldwide.
Making an impact globally
Since the launch of the Oxford Impact initiative in 2016, we have completed 40 impact studies.
A novel approach
In addition to the impact studies carried out after the launch of a product, OUP commissioned the University of Southampton to carry out an ‘upstream’ (prior to launch) impact evaluation in 2018. It is the first impact study (that we know about) concerning a feature, in this case prototype Coaching eBooks, prior to the launch of a service: Oxford Reading Buddy* (ORB). The research question was: ‘What impact can the use of the prototype Coaching eBooks have upon the reading attitudes and reading activities of pupils in Years 1 and 5?’ The research team guided OUP through this process and, at the end, both parties reflected in depth on the collaboration.
This evaluation aimed to inform the development of ORB by documenting how the prototype Coaching eBooks could be implemented in schools, while assessing their potential impact via a RCT over the 2018 summer term. The research team administered questionnaires that assessed pupils’ reading activities and attitudes to reading, before and after the introduction of the prototype Coaching eBooks. They also carried out teacher interviews and observed classrooms to produce a detailed understanding of how the Coaching eBooks were implemented and used. Three schools were selected in the Southampton area, UK. The study involved 12 teachers, 322 children, 6 Coaching eBook classes, 6 control classes and 7 prototype Coaching eBooks. One of the Year 1 classes (5–6 year olds) and one of the Year 5 classed (9–10 year olds) used the Coaching eBooks, the others did not. The research lasted 6 months.
Key evaluation findings
The study found that the average pupil’s attitude towards reading was significantly greater if they had been in a classroom that had used the prototype Coaching eBooks during that term. The impact on reading attitudes was most noticeable amongst the older, Year 5 pupils. This was an unexpected – but very encouraging – finding, as motivating older children to read is a particular challenge for teachers.
There were a number of challenges and difficulties that arose during the impact study. Here are a couple of examples, but many more were noted, some of which were acted on immediately and some will inform future developments of the service.
|
Impact study finding |
Impact on the service, support/messaging |
| Children weren’t motivated to take the quizzes and the user journey from Coaching eBook to quiz was not smooth. |
Introduced a number of new features aimed at encouraging children to take the quizzes, e.g. ‘I’ve done my First Quiz’ badge. Screen at the end of each eBook in which the Buddy encourages the child to take the quizzes. |
| Children weren’t motivated to open the hotspots (activation of the Reading Buddy) in the Coaching eBooks as frequently as hoped. |
Introduced an overview from the child’s Reading Buddy as part of a child’s ‘first time here’ experience, highlighting the purpose of hotspots in coaching texts. Highlighted importance to teachers in the User Support site of opening hotspots. Prioritized further research on engagement with the hotspots with our partnership schools. |
The partnerships that OUP forms around the world with educational research partners not only enable us to gather evidence on the impact of our resources, but can also help to facilitate educational policy change on a truly global scale.
Footnotes:
This blog is based on a presentation delivered at the recent International Congress for School Effectiveness and Improvement: ICSEI 2020.
More about the Oxford Impact initiative and a selection of impact study summary reports can be accessed here: Oxford Impact website.
*The Oxford Reading Buddy service has now launched in the UK, South Africa, Australia and across Asia. It won the Educational Publishing Awards Australia 2019. Key findings and details of the impact study and the academic report, written by researchers from the University of Southampton, UK, can be accessed here.
About the author:
Dr. Penelope Woolf is Head of Impact, Oxford Impact at Oxford University Press and Honorary Norham Fellow at Oxford University Department of Education. Dr. Woolf is passionate about the evidence-based agenda in education and ensuring that publisher’ resources are designed and developed to ensure that they “work”. Dr. Woolf can be reached at penelope.woolf@oup.com; penelope.woolf@gmail.com
Published by