On May 18, ANQA organised a webinar "Self Audit of Academic Programmes: Achievement and Verification of Outcomes". The meeting was organised to discuss and disseminate HEIs' practices as part of the "Self-Audit of Academic Programmes" project.
The webinar focused on how HEIs can ensure that students achieve all the outcomes established by an academic programme (AP), rather than only separate skills.
Self-audit as a tool for the evaluation of outcomes, rather than documents
The discussion highlighted that the AP self-audit should go beyond the description of existing documentation and focus on the actual achievement of the AP outcomes, the supporting evidence and the enhancement actions derived from the analysis.
Through this approach, self-audit connects outcome definition, teaching and assessment methods, data collection, analysis and further development.
From outcome mapping to the demonstration of student progress
The French University in Armenia shared its practice in the development of a QA system through outcome-based planning of APs, data management and self-audit.
It was highlighted that outcome mapping is more than a technical exercise. It should demonstrate:
- the subjects that contribute to the achievement of a specific outcome;
- the learning stages through which students' competences are developed;
- the progress through the beginner, intermediate and advanced levels;
- the assessment formats applied to verify the acquisition of those competences;
- the approaches used to minimise content duplication in the AP.
Through this logic, mapping is transformed from an additional document into a tool that demonstrates the AP structure, student progress and outcome achievement.
Combination of direct and indirect assessment for a comprehensive understanding
It was highlighted that the verification of the outcome achievement should not rely on a single tool or data source. Instead, it requires a combination of direct and indirect assessment mechanisms.
Direct assessment tools include examinations, project and group work, internships, graduation papers and theses. They enable the assessment of students' knowledge and competences through specific assignments.
Meanwhile, importance was attached to the indirect assessment tools, including student surveys, monitoring of graduates' professional progress, enhancement-focused recommendations and stakeholder feedback. These data provide insight not only into the assessment results, but also the impact of the AP on students' learning experience and professional development.
Focus groups: from gathering opinions to understanding competence development
The discussion also addressed the role of focus groups in the verification of the outcome achievement. It was highlighted that their value extends beyond the collection of general opinions and helps identify what learning experiences shape student competences. For instance, in the case of communication or negotiation skills, participation in a focus group alone is not sufficient to conclude that a student has acquired a particular skill. It is necessary to understand:
- the situations in which the student has applied the skill;
- the student behaviours;
- the issues the student has been able to solve;
- the student progress at different learning stages;
- the role of teaching and assessment methods in the development of the specific competence.
From this perspective, emphasis was placed on the use of situational and indirect questions, which make it possible to explore not only student opinions, but also their experiences, behaviours and competence development.
Data-driven QA as a basis for continuous enhancement
It was noted that the AP self-audit should be viewed as a continuous cycle, from definition and mapping of outcomes to data collection, analysis, planning of enhancement actions and monitoring of their implementation.
In this process, it is important that HEIs not only collect data but also analyse and apply it to support informed decisions on the AP development. The discussion also addressed the challenges faced by institutions, including low student participation in surveys, duplicated or conflicting data from various sources and ensurance of stakeholder engagement.
The achievement of learning outcomes should be reflected in the student development
The meeting concluded with an emphasis that each learning outcome should be viewed through a stage by stage development perspective, moving from basic levels towards more complex, applied and analytical levels. Only through such an approach can it be ensured that, upon completion of the academic programme, students have acquired not merely fragmented knowledge or isolated skills, but the comprehensive professional competences envisaged by the programme.
At ANQA’s initiative, such meetings will continue as part of AP self-audits, contributing to the exchange of emerging practices among HEIs.