Research | School Evaluation & Inspection | EQI
Inter Cultural Community Evaluation and Planning (ICCEP)
The project aims to develop a community-based planning and evaluation system, referred to as ICCEP where schools and other stakeholders collaborative in networks for the improvement of migration background students and a member of the network or an external body act as a facilitator and mediator during the ICCEP process. The term ICCEP means involving all stakeholders in the process of evaluation and planning. In this project, the education governance systems and the underpinning intercultural policies and documents of the participating countries will be reviewed. Teachers’, school leaders’ and other sectoral support agencies survey responses will lead towards developing a conceptual map of ICCEP. and the development of a framework of quality indicators and the detailed mechanism of ICCEP. Through case studies, the consortium will explore what training needs should be provided to upskill the existing competencies of members of the network. For this reason, this project includes developing a Massive Open Online Course (MOOC) and a toolkit of essential resources to plan for and integrate migrant students in communities. It will also involve training on the mechanisms required for ICCEP.
The project partnership is an experienced one and is made of representatives of Dublin City University, Johannes Kepler University, University of Oslo, Pamukkale University, Junta de Extremadura. Each partner brings interest, knowledge, experience and infrastructure to the project. Each partner country has an established system of quality assurance and educational governance and has a high proportion of migration background students in their education systems and therefore, they are interested in exploring the scope of ICEP as it specifically applies to all types of migrant students (refugees, asylum seekers and economic migrants) across Europe and developing a framework that offers a complete mechanism for this new model of evaluation along with a comprehensive professional development system for the upstream of stakeholder competencies in ICCEP. The envisaged outcome is that this project will provide a complete mechanism for ICCEP that are aligned with EU governance structures.
Project Coordinator: Dr Martin Brown
EQI Researchers: Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O'Hara, Ms Sarah Gardezi
DEAPS - Distributed Evaluation and Planning in Schools
DEAPS - Distributed Evaluation and Planning in Schools is a 3-year Erasmus+ funded project involving approximately 20,000 educational stakeholders throughout Europe. The overarching aim of DEAPS is to address issues of capacity in schools in the areas of school evaluation and quality assurance. It seeks to do this for the purpose of enhancing social inclusion and combatting issues such as Early School Leaving.
Based on an in-depth analysis of modes of evaluation DEAPS argues that the move from a centralized to distributed modes of evaluation is central to the inclusion of marginalized and under-represented voices in the quality assurance debate.
Arguing that greater inclusion will facilitate better outcomes DEAPS identifies the lack of capacity in schools to involve other voices– and in particular those of parents, pupils and management bodies - as being critical. To address this DEAPS will:
- Provide a conceptual and theoretical overview of Distributed systems of evaluation
- Develop a toolkit for schools targeted at developing capacity to include marginalized voices
- Develop a MOOC to facilitate distribution of information about DEAPS to non-school based actors (parents, management, etc.)
- Develop an initial teacher education module for use by HEI’s interested in including DEAPS approaches in their programmes
- Develop a project website to distribute all materials in a range of languages
The DEAPS partnership – made up of HEI’s from Ireland (Dublin City University Institute of Education), Turkey (University of Ankara), Belgium (University of Antwerp) and Portugal (Instituto Politécnico de Viseu) – is an experienced consortium of practitioners with a range of experiences in programme development, training, evaluation, early schools leaving and social inclusion at local, national and European level.
- Committed to a collegial and iterative process each partner will take responsibility for a number of identified outputs.
- Committed to transnational usage the products will be available in the 4 partner languages.
It is envisaged that the development of DEAPS will both enhance the capacity of schools to engage in inclusive quality assurance and will provide mechanisms that will allow groups who are in danger of exclusion find a meaningful voice in how schools judge themselves leading to greater engagement and ownership. This will, in the long term, transform the ways in which we consider what is important in education systems and bring inclusion and involvement to the centre or European education.
Partners: Centre for Evaluation, Quality and Inspection, DCU | University of Ankara | University of Antwerp | Instituto Politécnico de Viseu |
Project coordinator: Dr. Martin Brown
EQI Researchers: Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O'Hara, Dr. Shivaun O’Brien, Mr. Craig Skeritt
CRELES - Supporting Culturally Responsive Evaluation and Leadership in Schools
The context of this project relates to the increased immigration into countries of Europe with the consequent range of diversity of cultures among students in schools. In line with the current influx of migrants and refugees to Europe and the international Sustainable Development Goal of inclusive and equitable education for all, this project aims to prioritise the inclusive education of migrant students and the school leadership that facilitates that inclusive education.
This project focuses on the support and strategies needed in all levels of school leadership in order to facilitate the needs of a diverse multicultural classroom and a diverse multicultural school. The main objective of the project is the enhanced school inclusion of students with a migration background through the identification and dissemination of innovative practice in culturally responsive school leadership, the development of frameworks for the evaluation of culturally responsive leadership as well as the provision of high-quality training resources in the form of leader training and the development of a MOOC for school leaders and those interested stakeholder groups outside of the project.
Project partners come from Austria, Russia, Spain and Ireland
Project coordinator: Dr. Martin Brown
EQI Researchers: Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O'Hara, Ms. Sarah Gardezi
New Approaches in Inspection: A Polycentric Method
In recent years there has been significant interest in how we help schools improve and in particular how we assist good schools make the final transition to becoming schools that are characterized as ‘excellent’. Among the key stakeholders involved in this process are school inspectorates, many of whom have committed significant time and resources to helping schools make the make the leap from ‘good to great’ (McKinsey, 2010).
This project builds on the PINS – Polycentric Inspection of Networks of Schools (http://www.schoolinspections.eu/) project. EQI played a key role in this project and is now part of a group seeking to share insights and ideas on this innovative form of inspection with colleagues from Portugal, Turkey, The Netherlands and Bulgaria.
Project coordinator: Prof Joe O’Hara
EQI Researchers: Dr. Martin Stynes, Dr.Patrick Shevlin, Dr. Martin Brown, Professor Gerry McNamara, Ms. Sarah Gardezi, Mr Craig Skerritt
The Development of a Quality Framework for Ethos in Educate Together Schools
The Centre for Evaluation, Quality and Inspection (EQI) has been commissioned by Educate Together to develop a national Quality Framework for Ethos in Educate Together Schools. A key requirement for Educate Together is that the resulting Framework would support schools to self-evaluate and not place additional work on the schools beyond what was already required. As Educate Together expands the number of primary and second-level schools across Ireland, it has become more important than ever for schools to reflect on and strengthen how they implement this fundamental approach to teaching and learning and daily life of the school. Towards this end,
This work began in 2019 and involved widespread consultation with various stakeholders from within the Educate Together network. Interviews were conducted with key personnel in the national Educate Together office as well as a focus group with members of the Board of Directors. This led to the development of draft quality standards and statements of effective practice. Two national consultation meetings followed, in Cork and Dublin which was attended by representatives of 7 primary and 3 second-level schools. Participants in these full-day meetings included, students, teachers, Deputy Principals, Principals, parents and Board of Management representatives. The meetings involved active participation in a range of activities which encouraged attendees to focus on the meaning of ethos and to give feedback on the wording of the draft standards and statements of effective practice.
The Quality Framework involves setting out standards and statements of effective practice which schools will use as part of a self-evaluation process to evaluate practice, and to plan and implement improvements in the school. Draft guidelines for self-evaluation of ethos have been developed and are being tested by 21 Educate Together primary and post-primary schools across the country, over the 2019/20 school year.
This work is being led for DCU by Dr. Shivaun O’Brien, Centre for Evaluation Quality and Inspection and builds on almost 20 years of her experience in developing improvement frameworks and supporting schools and educational organisations to undertake self-evaluation.
Project Coordinator – Dr Shivaun O’Brien
Shaped Professional Learning Network – Northern Ireland
Developed by EQI senior research fellow Dr. Patrick Shevlin and Dr. Brown the primary focus of the EQI Shaped Professional Learning Network comprising of schools (cross phase – primary and post primary - controlled and maintained – across the religious divide, and including integrated and shared education) is to support targeted school improvement in a variety of forms; using newly developed and innovative techniques for continuing professional development and evaluation focused on a range of first hand evidence.
Those who were involved in the network do not see self-evaluation as being that of another initiative that has been bolted onto their increasingly heavy workload. Instead, self-evaluation is now viewed as a powerful and continuous process to improving the quality of teaching and learning in their schools. A clear but rigorous focus on quality learning as distinct from an undue focus on teaching was a key factor in the willingness of teachers to engage in the work of this network, often at their own expense and in their own time. In addition, school management released teachers from school to engage in the work, providing cover within their own schools and from their own school budget. There are clear messages therefore for the dissemination of this network practice to schools throughout Northern Ireland and indeed further afield.
A very effective way of professional learning has emerged in the production of these case studies which we characterize as a Shaped Network. On the one hand, those involved in the network could operate with a great deal of autonomy to choose their own focus for school improvement and associated school-based action research. On the other hand, they were also required to operate with a clearly defined focus using the school development plan as the foundations for the research and the rigorous use of first-hand evidence to measure impact.
There are over 100 schools currently involved in this network and it is intended to grow it in coming years.
Project coordinators: Dr Martin Brown, Dr Patrick Shevlin
ACRAS - Aiding Culturally Responsive Assessment in Schools - Erasmus+
Instead of a “one-fits-all” mentality ACRAS wants to develop methods of culturally responsive assessment which allow for differentiation and the contextualisation of learning in a culturally appropriate manner. Strategies which have previously been proposed for creating culturally responsive assessment include using locally validated formative assessments (Tichá & Abery, 2016), the addition of creativity assessments (see Kim & Zabelina, 2015) or utilising multiple methods of assessment to provide additional opportunities for students to demonstrate their learning (Castagno & Brayboy, 2008; Qualls, 1998). Yet no study, domestically or internationally, looks at the various strategies teachers use to integrate cultural responsivity into their student assessments and compare the relative merit of these strategies. As such, using a collaborative approach including a number of European HEIs, ACRAS will endeavour to develop a conceptual framework of best practice in educational assessment for minority and migrant students as well as supportive strategies (presented as a toolkit) to incorporate culturally responsive assessment in educators’ practice.
The first phase will include an initial exploratory survey in primary and post-primary schools (n≈120/country) to explore methods of assessment and any accommodations/differentiation used by educators for assessing migrant students. This initial survey will target the teachers and/or headteachers of these schools (n=480). Following the survey, approximately 12 schools per country will be chosen to participate in a case study. The case study will seek to elicit the experiences of all stakeholders (e.g. Principals, Class teachers, Students, Parents) using a multi-method approach (e.g. interviews, assessment biography). The case studies will aim to involve 4,224 participants across the four countries.
Based on the findings from the surveys and case studies, common strategies and pitfalls of assessment will be identified and collated into a conceptual framework of best practice in culturally responsive assessment. Based on the conceptual framework a toolkit will be established for educators. This toolkit will be piloted and, following alterations, externally evaluated. The toolkit has a potential legacy of at least 500 teachers (approximately 100 per year over 5 years) who complete Initial Teacher Education and Continuing Profession Development Courses in the Primary programme institution. These 500 teachers will be able to use the toolkit to influence the trajectory of their students' learning for years to come resulting in a potential impact of 5,000 minority and/or migrant students. EQI researchers include: Dr. Martin Brown, Prof Joe O’Hara, Prof. Gerry McNamara and Dr. Denise Burns.
Partners: Centre for Evaluation, Quality and Inspection, DCU | University of Oslo, Norway | JKU University, Austria | Cankiri Keratekin University, Turkey
EQI researchers include: Dr. Martin Brown, Professor Gerry Mcnamara, Professor Joe O'Hara, Dr. Martin Brown and Dr. Denise Burnes
PINS - Polycentric Inspection of Networked Schools - Erasmus+
In recent years there has been significant interest in how we help schools improve and in particular how we assist good schools make the final transition to becoming schools that are characterized as ‘excellent’. Among the key stakeholders involved in this process are school inspectorates, many of whom have committed significant time and resources to helping schools make the make the leap from ‘good to great’ (McKinsey, 2010) Among the approaches developed to facilitate this development various inspectorates of education (e.g. the Netherland and Northern Ireland) have begun to experiment with polycentric inspection. The underlying theory is that good schools can only further improvement not simply by pressure from external inspection but by collaboration between clusters of schools, communities and the inspectorate through the process of collaborative evaluation. Such polycentric inspection approaches should support schools in finding innovative ways to improve their teaching and learning; these approaches are expected to be more effective than centralized inspection standards and models which are thought to enhance and legitimize a ‘one size fits all’ strategy of success on one set of inspection standards and limit innovation (see Smith, 1995; Ehren et al in prep, O’Day 2002). This project proposes to evaluate the capability of polycentric inspection and collaborative self-evaluation as tools in improving schools from good to great in Northern Ireland, England, The Netherlands and Bulgaria.
Partners: Institute of Education, London | Sofia University, Bulgaria | University of Twente, The Netherlands | Centre for Evaluation, Quality and Inspection, DCU
EQI researchers include: Dr. Martin Brown Prof. Gerry McNamara, Prof. Joe O’Hara and Dr.Patrick Shevlin
E-Evalinto - Evaluation environment for fostering intercultural mentoring tools and practices at school - Erasmus+
Led by the University of Salamanca (Spain), this is a collaborative project involving DCU (Ireland), Oxfam Italia (Italy), University of Cadiz (Spain), SAN (Poland) and CARDET (Cyprus). E-Evalinto is a follow-up to an earlier European-funded project entitled INTO: Intercultural mentoring tools to support migrant integration at school.
The aim of E-Evalinto is to address the problem of early school leaving amongst refugee / migrant students at post-primary level by promoting intercultural mentoring programmes and activities supported by an ICT environment. This ICT environment will be composed of a set of tools and practices that provide a framework for evaluating intercultural issues.
EQI researchers include: Professor Gerry Mcnamara, Professor Joe O'Hara and Dr. Bernadette Sweetman
The first pilot school to take part in Ireland is St. Paul's CBS, North Brunswick St. Dublin. The participating students produced a news item on the project for their school website http://www.thebrunner.ie/evalinto
Testing Models of Support and Capacity Building for School Self-Evaluation - EQI
Building on the research carried out in 2014/15, on supporting schools to engage in school self-evaluation, a new research project was launched in September 2016 to further explore models that support and build capacity for self-evaluation in schools. In 2014/15, Dr. Shivaun O’Brien tested out a model of support in 5 post-primary schools. This model involved providing the support of an external facilitator to each school to guide SSE teams through the process. The resulting research demonstrated that each participating school completed a full cycle of the SSE process as intended and within a school year including gathering and analysis of data, the completion of an SSE report and a school improvement plan. The feedback from all schools was extremely positive with all teams claiming that the model of support resulted in an SSE process was easier to engage with and resulted in a range of task and process outcomes.
A new model of support was tested in 2015/16 and again in 2016/17. This model is based on the facilitated process but this time it attempts to support not just 5 but up to 15 schools per year in the process while at the same time building SSE capacity within each participating school. The research project is entitled Let’s help you to lead school self-evaluation in your school and involves the provision of a course to SSE leaders in order to support them, in turn, to facilitate SSE in their own school.
Course description: The course aims to support schools through the school self-evaluation process over the course of a school year. The course is geared for those who have responsibility for leading this process in the school. The course helps participants to get started in September and will outline a clear plan for how schools may complete the process by May. Participants will be supported one step at a time. Each session prepares the participant to implement the next stage of the process back in their own school. The model encourages wide consultation with the school community and engagement by stakeholders in the process.
The content of sessions will include the following:
- Getting started with SSE, establishing an SSE team, organising tasks and identifying what data to gather.
- Preparing questionnaires/ survey tools using Google Forms
- Collation and analysis of data
- Completion of the SSE Report
- Completion of the School Improvement Plan (SIP)
- Organising and Facilitating SSE Team meetings
EQI researchers include: Dr. Siobhan O'Brien, Professor Gerry McNamara, Professor Joe O'Hara and Dr. Martin Brown
Evaluation of the PDST National project for the integration of ePortfolios in Secondary Level Education in Ireland - PDST
The Centre for Evaluation, Quality and Inspection (EQI) has been asked by the Professional Development Service for Teachers (PDST) to evaluate their National E-portfolio project.
The use of ePortfolios to enhance the quality of education has become increasingly important in Europe and elsewhere. Indeed, in the case of Ireland, there is real commitment to explore ways in which ePortfolios can enhance teaching and learning across the continuum of education. At a policy level Digital Strategy for Schools 2015 -2018 (Department of Education, 2015) outlines a range of engagements with e-portfolios including the provision of online training on ePortfolios and assessment to all registered teachers in Ireland through teachercpd.ie and participation in a major EU-funded project, Eufolio.
In addition, as part of the Department of Education’s ongoing commitment to exploring the use of ePortfolios in Education, PDST - Technology in Education are testing a model of ePortfolio support at transition year level with Post Primary schools throughout Ireland. The support provided to participating schools involves capacity building through online training, face to face workshops and follow-up visits to ePortfolio schools at various junctures within the lifecycle of the project.
Within this, EQI has been commissioned by PDST to evaluate this National ePortfolio project. EQI has extensive experience of evaluating a wide range of innovative educational interventions across a range of settings and their particular expertise in programme evaluation will be central to this project. The principal EQI project output report will provide a mixed methods evaluation and assessment of a number of thematic areas including an extensive literature review of ePortfolio purpose, policy and practice in compulsory level education and an evaluation of the various stages of the PDST Transition Year Pilot Project.
Finally, it is envisaged that the EQI evaluation report on ePortfolios will allow various stakeholders such as the Department of Education, sectoral support services, Initial Teacher and Further Education providers and other interested organisations to enhance their understanding of the issues, mechanisms, training supports, and the value added attached to the systemic introduction of ePortfolios in Irish education. EQI researchers include: Prof Joe O’Hara, Dr. Martin Brown, Prof. Gerry McNamara and Dr. Shivaun O’Brien.
Partners: Centre for Evaluation, Quality and Inspection, DCU |Professional Development Service for Teachers
EQI researchers include: Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O'Hara and Dr. Siobhan O'Brien
Investigation of Teacher Education Tutors’ practice in ICT on the Island of Ireland - SCOTENS
A joint North-South (SCoTENS-funded) project entitled Teacher Education Tutors’ Practice in ICT. The research aims to explore how Initial Teacher Education (ITE) tutors cope with the challenges and expectations of the 21st-century learner in ITE and how they manage their own technological and pedagogical development as a lecturer.
The project has two phases:
(1) an initial online survey of teacher educators’ readiness to adapt and innovate in their teaching;
(2)an ethnographic study of primary and post-primary teacher educators’ practice in preparing student teachers for the profession.
Partners: University of Ulster | Queens University | Centre for Evaluation, Quality and Inspection, DCU
EQI researchers include: Professor Joe O'Hara, Dr. Martin Brown and Professor Gerry McNamara
Notes from a small Island – Co-existence revisited - EQI
Within the field of school evaluation policy and practice, finding a balance between school inspection and school self-evaluation has become a topic of much debate. Indeed, with the initiation of inspectorate devised frameworks for school self-evaluation in Northern Ireland and the Republic of Ireland; the co-existence of school Inspection and school self-evaluation has shifted from a preponderant theoretical lens to that of a regulatory requirement where schools are now required to engage with an externally devised process of school self-evaluation that is subsequently quality assured by the inspectorate. This research project provides an analysis of the changing landscape of school evaluation policy and practice on the island of Ireland and also, aims to explore both internal and external challenges relating to the co-existence of such a relationship.
National Survey of Principals Attitudes Towards School Self – Evaluation (2014-2015)
EQI researchers include: Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O'Hara, and Dr. Siobhan O'Brien
Facilitating School Self-Evaluation - EQI
School Self-Evaluation (SSE) was introduced into Irish schools in 2012 as a mandatory requirement. A range of supports have been provided to schools by the Department of Education and Science inspectorate as well as the Professional Development Service for Teachers (PDST), in this regard.. Such supports involve the provision of guidelines and training for school personnel which is similar to supports provided across other OECD countries. However, many schools are experiencing difficulty engaging in the SSE process due to pressures of time and the lack of in-school supports for coordinating the SSE process. As a result many schools have not yet engaged in SSE or have engaged to a limited degree.
This research projects aims to explore a particular approach to supporting post-primary schools to engage in school self-evaluation. The model of support involves the provision of an external professional to guide and facilitate the process of SSE in 6 post-primary schools over the duration of 1 school year (2016-2017). The project will attempt to identify a model of support that is both cost-effective and efficient ( achieving both task and process outcomes) and one that may usefully be applied nationwide.
Baseline data gathered at the start of the project, identifies previous engagement of the 6 schools in SSE, the nature of the supports accessed by schools in this regard and the attitudes of Principals towards the SSE process. After applying the model of support in each of the 6 schools, the outputs and experience of the process will be evaluated. A revised model of support will then be developed which may have an application nationwide.
EQI researchers include: Dr. Siobhan O'Brien, Professor Gerry McNamara, Professor Joe O'Hara and Dr. Martin Brown
EQI is a multidisciplinary research group that seeks to design, implement and assess the impact of evaluation and inspection in education and related fields. It also has a strong commitment to engaging in research into evaluation as theoretical field and to exploring the impact of changing conceptions of value, quality and achievement within education systems. It also promotes evaluation studies within the University and continues to support undergraduate and postgraduate students who wish to engage in research in the field.
At a fundamental level EQI sees research as providing a foundation for its work. It is an active partner in local, national and European projects and continues to seek to find ways to influence policy and practice in education and related fields within Ireland and beyond.