Formative Assessments Co-created with Students (FACCtS)

This project has produced the following major outputs.
The two major contributions of this project are: First, it facilitated an improved learning experience for the students by building and offering highly interactive and engaging formative assessment questions with immediate feedback. As a result of this project:

1. Students and the module coordinator co-created more than 350 FACCtS that include several question types, including MCQ, T/F, Matching, Fill the Blank, Drag and Drop, Flip Card, etc. These FACCtS covered the seven microcurricula that are included in the module.

2. FACCtS are implemented in the module's Loop page using a highly interactive design and tool called H5P. The FACCtS were presented to students as optional questions at the end of each microcurricula, as a means of checking their readiness to take the final assessment.

3. These questions were made available to more than 2190 students in the 2024/25 academic year. Although these questions are optional, up to 25% of the students were fully engaged and completed these optional questions, leading to an improvement in the summative assessment results of participating students.

Second, it offered opportunities for students and the module coordinator to shape the future of the module. As a result, the following experience is gained.

1. The role of current and previous students in shaping the future of their peers’ learning experience is significant. As supported by the constructivist learning theory, the idea that students actively construct knowledge through meaningful interactions with content, peers, and instructors is crucial in their learning; thus, curriculum designers should engage with students at some stage of the design process.

2. Designing interactive and UDL-compliant modules that are delivered to very large classes in an online and asynchronous environment is challenging. However, careful alignment involving students can leverage the effort to provide students with a meaningful experience.

3. Where there is little opportunity to provide meaningful feedback, particularly in an asynchronous environment, FACCtS could fill the gap by enabling high-quality and immediate feedback.

Based on a mixed-method research I conducted on the impact of FACCtS, the following impacts were observed.

1. FACCtS Improved Student Performance. Quantitative results show that FACCtS improve the performance of students in five of the seven microcurricula. Statistically significant improvement was observed in two microcurricula.

2. FACCtS Improved Student Engagement:  FACCtS improves the engagement of students in all seven microcurricula, with a statistically significant increase in engagement in four. This means that students who participated in the formative assessment engage in further attempts at the summative assessment. There is a statistically significant increase in the engagement of students with the summative assessment questions.

3. FACCtS Enhanced Active Learning Engagement. One of the primary impacts of FACCtS is further transforming the online learning environment into a more interactive and motivating experience through the inclusion of several elements that are relevant to the students’ learning experience. These elements include highly interactive questions that include diversified sets of MCQ, True/False, drag-and-drop and flip-board type questions that engage students to continue answering questions.

4. FACCtS Enabled Metacognitive Learning Development. Another theme that emerged is that FACCtS positively impact the metacognitive learning development of the students. The FACCtS served as a mirror for student learning and reflection and supported adjustment of their learning strategies. The students were able to develop sophisticated awareness of their knowledge gaps and learning progress, enabling them to make informed decisions about their own study strategies and readiness for summative assessments.

5. FACCtS Enhanced Applied Learning Integration. Another impact demonstrated by FACCtS is their ability to effectively bridge the gaps between theory and practice by offering students opportunities to practically apply the theoretical lessons in practice. Students have reported the use of FACCtS content in assignments, projects and subsequent courses, indicating successful knowledge transfer. This is not solely attributed to FACCtS, but generally to the DLA module. FACCtS focuses on theoretical concepts, repeatedly engages students in different real-world use cases and encourages the application of the theories into practice. Students used the lessons learned from FACCtS to complete their assignments and group projects.

Key Learning 

Maintain Co-Creation Approach. The appreciation and recognition of FACCtS suggest this approach should be preserved and potentially expanded by including additional students to contribute to further enhancement of the module. Students value the authentic perspective that peer creators bring to question design.

Enhance Feedback Systems. As immediate feedback plays a significant role in driving learning loops, investment in more sophisticated feedback mechanisms could amplify learning benefits. Students suggested features like direct links back to relevant content sections for incorrect answers. This could be further achieved with automated tools that could relate the answers to the sections that are directly related to the questions.
Support Further Personalisation. The sophisticated self-regulation strategies students developed suggest that the flexible, self-paced structure is crucial for effectiveness. Maintaining this flexibility while providing optional structure for students who need it could optimise outcomes. This includes expedited learning paths enabling students with prior knowledge to quickly verify their knowledge and go directly to the summative assessment questions.

Address Cognitive Load: Students noted challenges with content volume and text-heavy materials. Incorporating more multimedia elements and managing information density could enhance engagement and reduce cognitive overload. Along with this, a co-occurring concept, studying at the last minute, contributed to the huge cognitive load. A potential solution should be included to encourage students to engage with the content as early as possible to avoid the potential cognitive load in completing FACCtS and the summative assessment pieces.

Consider Progressive Complexity. Students noted increasing difficulty levels, which appear to support learning progression. Careful calibration of progression to complex concepts could optimise challenge levels throughout the learning sequence.

Future Work

As part of a follow-up SATLE funding, this approach is expanded to cover the remaining microcurricula. Current work is underway to co-design Python and R routes to provide similar opportunities for students who opt for Python and R. A comparative analysis using mixed methods research has been completed to assess the impacts of FACCtS on the learning experience of students.

The lessons learned from this project and the currently ongoing implementation indicate that there are still questions that have not yet been answered by the current research. Future research in the area will consider the role of students in shaping their peers’ learning experience and developing better strategies to increase the participation of students in completing FACCtS.

For programs that plan to adopt online asynchronous delivery of their modules will benefit from the educational strategy followed to create FACCtS. The methodology allows module coordinators to invite high-achieving students to share their experiences in the form of questions, peer-learning activities, and module summaries. Furthermore, the evaluation offers a rigorous method to assess the impact of FACCtS on the learning experience of students.

Evaluation

The research conducted in this area focuses on evaluating the impact of several FACCtS part of the enrichment introduced to a university-wide Data Literacy and Analytics (DLA) module delivered to students from 14 programmes in an online and asynchronous environment at Dublin City University. This enrichment, among other things, mainly focuses on co-creating formative assessment questions with students to enhance the learning experience of future students enrolled on DLA for the 21st Century module (CSC1015). More than 350 formative assessment questions were co-created with students who completed the module in the previous academic year, and implemented into the 2024/25 instance of the module, which is delivered to more than 2190 students. Thus, this research investigates the impact of this enrichment on the learning experience of the students, focusing on their performance in the summative assessment, the number of attempts they made to pass the module, and their overall experience in completing FACCtS. This research uses a pragmatic paradigm to answer the research questions. A mixed method is used as a research method to understand the phenomenon. Quantitative data collected via Loop is used to answer the primary question using objective data capturing the interaction of the students with the module content throughout the academic year. To further strengthen the results obtained from the quantitative data, qualitative data is collected using interviews. Statistical analysis and thematic analysis are applied to answer the research questions.