☰       Department Menu

Centre for Assessment Research, Policy and Practice in Education (CARPE)


Publications

Below you can view publications and other scholarly outputs that have arisen from CARPE's various research projects.   

 Peer-Reviewed Journal Articles 

2020

Darmody, M., Lysaght, Z. & O'Leary, M. (2020). Irish post-primary teachers’ conceptions of assessment at a time of curriculum and assessment reform. Assessment in Education: Principles, Policy & Practice.

__________________________________________________________________________________________

2019

Lehane, P. (2019). What Makes a Difference for Candidates Taking Computer-based Tests? Issues Surrounding Device Comparability and User Interface Modifications. CLEAR Exam Review

Scully, D., Pitsia, V. & Karakolidis, A. (2019). Exploring the interpersonal dimension of teaching in an Irish post-primary context. Irish Educational Studies, DOI: 10.1080/03323315.2019.1697947 

__________________________________________________________________________________________

2018

Scully, D., O'Leary, M. & Brown, M. (2018).  The learning portfolio in higher education: A game of snakes and ladders (Junhong Xiao trans.) Distance Education in China.  DOI: 10.13541/j.cnki.chinade.20180509.001 

O'Leary, M., Scully, D., Karakolidis, A. & Pitsia, V. (2018).  The state-of-the-art in digital technology-based assessment.  Eur J Educ., 53, 160-175. 

__________________________________________________________________________________________

2017
Lysaght, Z., & O’Leary M. (2017). Scaling up, writ small: using an assessment for learning audit instrument to stimulate site-based professional development, one school at a time. Assessment in Education: Principles, Policy and Practice, (24)2, 271-289.

Lysaght, Z., O'Leary, M. & Scully, D. (2017).  Irish Pre-Service Teachers' Expectations for Teaching as a Career: A Snapshot at a Time of Transition.  Irish Journal of Education, 42 

Lysaght, Z. O'Leary, M. & Ludlow, L. (2017).  Measuring Teachers' Assessment for Learning (AfL) Classroom Practices in Elementary Schools.  International Journal of Educational Methodology, 3 (2), 103-115. 

Karakolidis, A., O'Leary, M. & Scully, D. (2017).  Practice tests: do they work?  Can they work better?  CLEAR Exam Review, 27 (1)  

Pitsia, V., Biggart, A. & Karakolidis, A. (2017). The role of students’ self-beliefs, motivation and attitudes in predicting mathematics achievement:  A multilevel analysis of the Programme for International Student Assessment data.  Learning and Individual Differences, 55, 163-173. 

Scully, D. (2017). Constructing Multiple-Choice Items to Measure Higher-Order Thinking. Practical Assessment, Research & Evaluation22 (4). Available online: http://pareonline.net/getvn.asp?v=22&n=4
_______________________________________________________________________________________________________

2016 
Karakolidis, A. Pitsia, V. & Emvalotis, A. (2016). Examining students’ achievement in mathematics: A multilevel analysis of the Programme for International Student Assessment (PISA) 2012 data for Greece.  International Journal of Educational Research, 79, 106-115

 Conference Presentations 


2020

Kuan, L., O'Leary, M. & Brunner, B. (2020). Is the Psychometric Quality of Situational Judgment Items Affected by the Type of Task Analysis Performed? Paper accepted for presentation at the  Association of Test Publishers (ATP) Conference, San Diego, United States. (Conference cancelled).

Lehane, P., Lysaght, Z. & O'Leary, M. (2020) The Interview as a Selection Mechanism for Entry into Initial Teacher Education: A Review of the Literature and Recommendations for Practice. Paper accepted for presentation  at the Educational Studies Association of Ireland, Dublin, Ireland. (Conference cancelled).

Lehane, P., Pitsia, V., & Karakolidis, A. (2020). Identifying factors predicting teachers’ use of assessment data: Findings from a national large-scale survey of primary teachers in Ireland. Paper accepted for presentation at the European Conference on Educational Research 2020, Glasgow, UK. (Conference cancelled).

Ling, G., O'Leary, M. et al. (2020). Assessment of Competencies as a Result of College Learning: Explorations in Europe and Beyond. Symposium accepted by Division 9 (Assessment, Evaluation, Testing and Measurement) of the European Conference on Educational Research 2020, Glasgow, UK. (Conference cancelled)

O'Leary, M., Lysaght, Z., NicCraith, D. & Scully D. (2020). Teacher Perspectives on Standardised Testing of Achievement in Ireland. Paper accepted for presentation at the Annual Meeting of the American Educational Research Association (AERA), San Francisco, United States. (Conference cancelled).

O'Leary, M. & Reynolds, K. (2020). Critical Thinking Test Validity in International Contexts: Evaluating the ETS HEIghten Test for Irish Use. Paper accepted for presentation at the Annual Meeting of the American Educational Research Association (AERA), San Francisco, United States. (Conference cancelled).

Pitsia, V., O’Leary, M., & Shiel, G. (2020). A multilevel binary logistic regression analysis of mathematics and science achievement in TIMSS 2015 in an Irish post-primary context. Paper accepted for presentation at the European Conference on Educational Research 2020, Glasgow, UK. (Conference cancelled).

Pitsia, V. (2020). Characteristics of high achievers: A multilevel logistic regression analysis of PISA mathematics and science data. Paper accepted for presentation at the American Educational Research Association (AERA) 2020 Annual Meeting, San Francisco, United States. (Conference cancelled).

Pitsia, V., Lysaght, Z., Shiel, G., & O’Leary, M. (2020). Are we meeting the needs of high achievers? A closer look at PISA, TIMSS and PIRLS data for Ireland. Paper accepted for presentation at the Educational Studies Association of Ireland Conference 2020, Dublin, Ireland. (Conference cancelled).

Reynolds, K., Abrams, L. & O'Leary, M. (2020). Applying the Webb Alignment Model to Professional Testing. Paper accepted for presentation at the  Association of Test Publishers (ATP) Conference, San Diego, United States. (Conference cancelled).

Scully, C. (2020). Inter-rater reliability and validity in the Objective Structured Clinical Examination. Paper accepted for presentation at the Educational Studies Association of Ireland, Dublin, Ireland. (Conference cancelled).

_______________________________________________________________________________________________________________

2019

Clifford, I., & Karakolidis, A. (2019, September) The challenges and opportunities of including animated items in licensure examinations: Insights from two research studies. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Madrid, Spain.

Karakolidis, A. (2019, April). The use of animations in assessment: Comparing an animated and a text-based situational judgment test. Paper presented at the American Education Research Association (AERA) Annual Meeting. Toronto, Canada.

Karakolidis, A., O'Leary, M., & Scully, D. (2019). Enhancing assessment validity through the use of animated videos: An experimental study comparing text-based and animated situational judgements tests. Paper presented at the 20th annual Association for Educational Assessment – Europe (AEA) conference. Lisbon, Portugal.

Lehane, P. (2019, November). Designing Digital Assessments: Factors to Consider when Developing Computer-Based Exams. Paper presented at the World Conference of Online Learning (WCOL) conference, Dublin, Ireland.

Lehane, P. (2019, September). Considering ‘Emotional Intelligence’ in Certification and Licensure Testing. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Madrid, Spain.

Lehane, P. (2019, April). Assessment 2.0: Factors to consider when developing technology-based assessment. Paper presented at the Educational Studies Association of Ireland (ESAI) conference, Sligo, Ireland. 

Lehane, P. & Kuan, L. (2019, March). Finding a suitable task analysis for developing competency-based statements. Paper presented at the  Association of Test Publishers (ATP) Conference, Orlando, United States.

Lehane, P. & Waters, L. (2019, March).  What makes a difference for candidates taking computer-based tests? The impact of devices and user interface tools. Paper presented at the  Association of Test Publishers (ATP) Conference, Orlando, United States.

O'Leary, M. & Champagne, K. (2019, March). Alternative and high-fidelity item types. Where are we now with technology-based items? Paper presented at the  Association of Test Publishers (ATP) Conference, Orlando, United States.

O’ Leary, M., Scully, D. & Lehane, P. (2019, September). Considering emotional intelligence in certification and licensure testing. Paper presented at the Association of Test Publishers (ATP) Public Sector Special Interest Group Conference, Vienna, Austria.

Pitsia, V., Karakolidis, A., & Shiel, G. (2019, June). High achievement in mathematics and science: A multilevel analysis of TIMSS 2015 data for Ireland. Paper presented at the 8th International Association for the Evaluation of Educational Achievement (IEA) International Research Conference (IRC-2019), Copenhagen, Denmark.

Scully, D. (2019, January). The Learning Portfolio in Higher Education: A Game of Snakes and Ladders? Paper presented at the Middle East and Africa Association of Test Publishers (MEA-ATP) Conference, Abu Dhabi, United Arab Emirates.

Scully, D. & Kuan, L. (2019, January). Providing useful and effective diagnostic feedback. Paper presented at the Middle East and Africa Association of Test Publishers (MEA-ATP) Conference, Abu Dhabi, United Arab Emirates.

Scully, D., O'Leary, M. & Lehane, P. (2019, September). Considering emotional intelligence in certification and licensure testing. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Madrid, Spain.

_______________________________________________________________________________________________________________

2018 

Karakolidis, A. & Corrigan, R. (2018, February). Is a picture worth a thousand words? Using animated items in certification testing. Paper presented at the Association of Test Publishers Conference, San Antonio, TX.

Karakolidis, A. & Pitsia, V. (2018, September). Transforming Situational Judgement Tests through the use of animated simulations. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Athens, Greece.

Lehane, P. (2018, September).  Device Comparability – Does it matter what device is used to administer assessments?  Paper presented at the European Association of Test Publishers (E-ATP) Conference, Athens, Greece 

Lysaght, Z. & Scully, D. (2018, September). Providing Diagnostic Feedback to Non-Successful Test Candidates. Paper presented at the Association of Test Publishers (ATP) Public Sector Special Interest Group Conference, Brussels, Belgium.

Scully, D. & Kuan, L. (2018, September). Identifying the Task Analysis of Best Fit. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Athens, Greece.

Scully, D. & Lysaght, Z. (2018, September). Diagnostic Feedback – Dos and Don’ts. Paper presented at the European Association of Test Publishers (E-ATP) Conference, Athens, Greece. 

Scully, D., O’Leary, M. & Brown, M. (2018, August). Key Findings from a Literature Review on the Use of Learning Portfolios (ePortfolios) in Higher Education. Paper presented at the World Education Research Association (WERA) World Congress, Cape Town, South Africa.

Scully, D., O'Leary, M. & Brown, M. (2018, June).  The Learning Portfolio in Higher Education: An Integrative Review.  Paper presented at the EdMedia + Innovate Learning International Conference, Amsterdam, Netherlands.  

Scully, D., O'Leary, M. & Brown, M (2018, May).  A Game of Snakes and Ladders: An Integrative Review of Learning Portfolio in Higher Education.  Paper presented at the Irish Learning Technology Association EdTech Conference, Carlow IT. 

Scully, D., O'Leary, M. & Brown, M. (2018, May).  The Learning Portfolio in Higher Education: A Game of Snakes and Ladders? Paper to be presented at the CRA/AAEEBL International Seminar, ePortfolios and more: the developing role of ePortfolios within the digital landscape, Dublin City University.  

Scully, D. & Ridgley, K. (2018, February). Competencies - What are they and how best to we define them to measure them?  Paper presented at the Association of Test Publishers Conference, San Antonio, TX. 

Stemler, S., Elliott, J., O'Leary, M., Scully, D., Karakolidis, A. & Pitsia, V. (2018, April).  A Cross-Cultural Study of High School Teachers' Tacit Knowledge of Interpersonal Skills.  Paper presented at the Annual Meeting of the American Educational Research Association (AERA), New York, NY. 

_______________________________________________________________________________________________________________

2017 
O'Leary, M., Scully, D. & Karakolidis, A. (2017, September).  Refining Situational Judgement Tests.  Paper presented at the European Association of Test Publishers Conference, Noordwijk, The Netherlands.  

O'Leary, M. Scully, D. & Karakolidis, A. (2017, September). Challenging a Tenet of Multiple Choice Testing: Are Four Response Options Really Necessary?  Paper presented at the European Assoication of Test Publishers Conference, Noordwijk, The Netherlands. 

Gurhy, AM (2017, April). Using assessment for learning to enhance mathematics education in the primary school: A lesson study approach.  Paper presented at the Annual Conference of the Educational Studies Association of Ireland Conference, University College Cork.  

Karakolidis, A., Scully, D. & O'Leary M. (2017, March).  Simulations in Assessment and the Uncanny Valley: Too true to be good?  Paper presented at the International TEchnology Education and Development (INTED) Conference, Valencia 

Scully, D. & Waters, L. (2017, March).  Driving Program Decisions Through Research: Applying Best Practices.  Paper presented at the Association of Test Publishers Conference, Scottsdale, AZ.  

Gurhy, AM (2017, February).  Using assessment for learning to enhance mathematics education in the primary school: Irish students' perspectives.  Paper presented at the 10th Congress of European Research in Mathematics Education (CERME 10), Croke Park, Dublin.
_______________________________________________________________________________________________________________ 

2016 
Lysaght, Z. & O'Leary, M. (2016, August).  Developing Assessment Capacity in Norway and Ireland using the Assessment for Learning Audit Instrument (AfLAi).  Paper presented at the Annual Meeting of the European Conference for Education Research (ECER), Dublin. 

O'Leary, M. & Lysaght, Z. (2016, April).  Using the Assessment for Learning Audit Instrument to inform Data-driven Professional Development.  Paper presented at the Annual Meeting of the American Educational Research Association (AERA), Washington, D.C.   

Reports

2020

Butler, D., McLoughlin, E.,O'Leary, M., Kaya, S., Costello, E., & Brown, M. (2020).  Towards the ATS STEM Conceptual FrameworkATS STEM Report #5. Dublin: Dublin City University. 

Reynolds, K., O’Leary, M., Brown, M. & Costello, E. (2020). Digital Formative Assessment of Transversal Skills in STEM: A Review of Underlying Principles and Best Practice. ATS STEM Report #3. Dublin: Dublin City University.

Szendey, O., O'Leary, M., Scully, C., Brown, M., & Costello, E. (2020). Virtual Learning Environments and Digital Tools for Implementing Formative Assessment of Transversal Skills in STEM.  ATS STEM Report #4. Dublin: Dublin City University

_______________________________________________________________________________________________________________

2019

Lehane, P. (2019). Leaving Certificate Computer Science: Factors to consider when developing a computer-based examination. National Council for Curriculum and Assessment and Centre for Assessment Research, Policy and Practice in Education (CARPE)

Lysaght, Z., Scully, D., Murchan, D., O’Leary, & Shiel, G. (2019). Aligning assessment, learning and teaching in curricular reform and implementation. National Council for Curriculum and Assessment: Dublin.

O’Leary, M., Lysaght, Z., Nic Craith, D., & Scully, D. (2019). Standardised Testing in English Reading and Mathematics in the Irish Primary School: A Survey of Irish Primary Teachers. Dublin: Dublin City University, Centre for Assessment Research Policy and Practice in Education and Irish National Teachers’ Organisation. 

_______________________________________________________________________________________________________________

2018 
O'Leary, M. & Scully, D. (2018).  The Leaving Certificate Programme as Preparation for Higher Education: The Views of Undergraduates at the End of their First Year in University.  Dublin: Dublin City University, Centre for Assessment Research, Policy & Practice in Education (CARPE). 

Scully, D., O'Leary, M. & Brown, M. (2018). The Learning Portfolio in Higher Education: A Game of Snakes and Ladders.  Dublin: Dublin City University, Centre for Assessment Research Policy & Practice in Education (CARPE) and National Institute for Digital Learning (NIDL).

Scully, D., O'Leary, M. & Brown, M. (2018).  El Portafolio de Aprendizaje en la Ensenanza Superior: Una Partida de Serpientes y Escaleras. Translated by DCU Language Services.  Dublin: Dublin City University, Centre for Assessment Research Policy & Practice in Education (CARPE) and National Institute for Digital Learning (NIDL).   

_______________________________________________________________________________________________________________

2017 
O'Leary, M. & Scully D. (2017). Providing effective feedback in the context of certification and licensure testing with particular reference to non-successful test candidates.  Unpublished manuscript, Centre for Assessment Research, Policy & Practice in Education, Dublin City University. 

Scully, D. (2017).  Reducing Gender Bias in Tests and Test Items.  Unpublished manuscript, Centre for Assessment Research, Policy & Practice in Education, Dublin City University.  
_______________________________________________________________________________________________________________

2016 
Scully, D. (2016).  Awarding Partial Credit in Multiple-choice Testing.  Unpublished manuscript, Centre for Assessment Research, Policy & Practice in Education, Dublin City University.  

Scully, D. (2016).  Some Recommendations for Item Development.  Unpublished manuscript, Centre for Assessment Research, Policy & Practice in Education, Dublin City University.