Programme Assessment

Getting started

We are now at the stage of developing a programmatic approach to assessment.

Process diagram for Programme Development stages and processes. The first is programme learning outcomes. The second, which is highlighted is Programme Assessments. Following on from here is Module Learning Outcomes, and Module Assessments and Learning Activities.

Why plan Programme Assessments?

Programme-level planning enables us to ensure that the student learning experience is one which builds throughout the programme, with assessment methods aligned to learning outcomes and learning activities. Taking a programmatic view promotes good assessment practice and can contribute to the development of a coherent and inclusive assessment strategy.

An infographic which is an extract from the UK Quality Code for Higher Education Advice and Guidance Assessment. Tick boxes appear next to the following titles, with supplementary detail beneath. Title 1: ASSESSMENT METHODS AND CRITERIA ARE ALIGNED TO LEARNING OUTCOMES AND TEACHING ACTIVITIES. More detail on Title 1: There is alignment between intended learning outcomes, teaching strategies, methods of assessment and assessment criteria. Title 2: ASSESSMENT DESIGN IS APPROACHED HOLISTICALLY. More detail on Title 2: Assessment is designed top down' - beginning with the award, then going down into module level. Title 3: ASSESSMENT AND FEEDBACK IS PURPOSEFUL AND SUPPORTS THE LEARNING PROCESS. More detail on Title 3: Assessment relates directly to course aims and learning outcomes.

Effective design of assessment ensures that programme learning outcomes (which themselves fulfil the requirements of the relevant level descriptors in the FHEQ and QAA Subject Benchmark Statements) are addressed through the assessment of the course and its constituent units or modules. In this toolkit we focus on the fundamentals of a programmatic approach to assessment. Essentially, our approach is:

A coherent programmatic approach to assessment is one where all elements of assessment and feedback (formative and summative) are strategically related with purposeful intent to achieve the programme learning outcomes.

However, in recognition of various interpretations of programmatic assessment we explore more ‘integrative’ and ’innovative’ levels in the ‘deeper dive’ section . The deep dive section also provides a series of reflective prompts and additional information for you to consider.

An infographic showing increasingly progressive and radical approaches to programmatic approaches to assessment. Fundamentals is first, represented by a map of all assessment points. The detail says: Ensuring a programme wide coherent approach to Assessment. Integrative is next, represented by a colourful diagram. The detail says: Incorporating aspects of synoptic/capstone assessments. Innovative is last, represented by a mapping of teaching to assessment points. The detail says: A longitudinal data point approach.

POINTS FOR REFLECTION:
In a nutshell, when you are seeking to plan for programme-level approaches to assessment, you should be clear how you intend to do this. The first question is more of a fundamental of quality education, whereas the second and third represent increasingly innovative approaches that will need careful consideration.

Are you simply going to ensure there is a good spread, balance and progression in assessment across your programme?
Will you demonstrate learning from across modules in a special assessment, such as a capstone?
Will you design an innovative approach to supporting and tracking learning?

Inclusivity Tip

To ensure that your programme is inclusive, consider the modes of assessment across your programme, as well as the content. A balance of assessment types (for example written, oral and practical) enables students with different skills and abilities to flourish.

Also consider progression: well-designed progression from low- to high-stakes assessment supports students in developing their skills. For more details, see our Inclusive Programme Design section, which considers Inclusive Assessment Strategy Considerations, and Inclusive Programme Assessment Mapping.

Baseline requirements of programmes

You will be expected to show a holisitic approach to assessment in your assessment mapping and programme delivery and assessment line plans.

Fundamentals of Programmatic Assessment

Having determined and agreed the intended learning outcomes as a foundation, the next stage is to start developing an approach to assessment across the whole programme.

Nothing you do at this stage will be ‘set in stone’, but by completing the below exercises you can develop a framework to guide assessment design across the programme; something you should revisit and seek to validate as you move through this process.

Balancing the purpose of Assessment Activities

It is important to determine the balance between assessment activities that are either ‘as learning’, ‘for learning’ and ‘of learning’. To learn more about these, explore the assessment section of the Programme Development page.

By designing an approach that balances the three main purposes of assessment you will be able to:

Ensure students understand how assessment works, what standards they should evidence in different pieces of work, better understand what marks mean and represent, and better understand feedback comments – ASSESSMENT AS LEARNING (AaL);
Test their understanding and skills in formative tasks, and more effectively utilise the feedback they will receive from such tasks – ASSESSMENT FOR LEARNING (AfL) and;
Be better prepared and able to submit work that represents their skills and achievements in summative tasks – ASSESSMENT OF LEARNING (AoL).

Activity
Meet as a programme team to discuss how you might plan a balance of these difference types of assessment in your programme.

Mapping of Assessment Methods against PLOs

The second stage of the process is creating the assessment methods that could be used to support student demonstration of the programme learning outcomes.

At this stage the intended programmatic assessment (IPA) methods are set at a high level, utilising categories of assessments that can be found on of Holistic Template in the Deeper Dive. Methods include exams, coursework, presentation, group-work and more. Determining the mix of methods best suited to the different learning outcomes indicates that an inclusive approach to assessment design is built into the programme.

The outputs from this stage will allow you to identify how student progress towards achievement of the outcomes can be built into programmes across the different levels of study. For example, in an undergraduate programme should you have a programme learning outcome (PLO) that is focussed on the accumulation and demonstration of communication skills, you may identify a presentation as a suitable intended programmatic assessment (IPA). In doing this it would make sense for students to be introduced to presentations as an assessment method at Level 4, have a further opportunity to practice these skills and utilise feedback already received at Level 5, and then to undertake a larger and summatively assessed presentation at Level 6.

The intention of the programmatic approach is to enable both cohesion yet also flexibility and innovation, therefore as you further develop your Module Assessment strategies you will be able to identify distinct sub-types of assessment that would best suit the modular level outcomes. This will again enhance variety and diversity of assessment and embed inclusivity in your assessment design.

Assessment strategy considerations

Effective assessment design is founded on a set of Cardiff University Assessment and Feedback Principles and Commitments, available here. These set out guidance and advice on how assessment and feedback will be designed and undertaken across the University. They aim to highlight common features of effective assessment and feedback and the actions and processes that ensure a good, consistent student experience.

Having given consideration to the Intended programmatic assessment methods and the balance of Aol, Afl and AaL you would then be able to map out an indicative programmatic assessment strategy, one that considers how the main intended assessments may apply across the levels of study.

For some practical tips for a coherent assessment strategy at a programme level, we suggest programmes plan their assessment to ensure alignment to learning outcomes, encourage assessment diversity, address volume and ensure balance of workload, embrace formative assessment and feedback as process, ensure assessments are scaffolded within and across levels of study and are authentic. At the heart of programmatic assessment sits an inclusive approach.

Thinking Programmatically is represented by an abstract pie chart in pastel colours. Inclusion is represented in the middle of the hollow pie chart by its toolkit symbol: a galaxy emoji. Alignment, diversity, balance, formative feedback, scaffolding and authenticity are all represented as an equal part of thinking programmatically.

Click Here For More Support on Effective Assessment Design

Alignment

Review the alignment between the programme learning outcomes and your module learning outcomes (see our LO mapping section for more info)
Consider how your assessment task meets individual programme learning outcomes

Diversity:

Ensure a diversity of assessment methods across the programme – see our compendium of assessment types
Where possible, take a use Universal Design for Learning to plan your assessments

Volume and Balance:

Aim to ensure your assessment volume is within the Cardiff University Tariff guidelines, available here. As a brief example see below guidance for traditional assessment types taken from the Guidance on the Nature and Volume of Assessment in Modules on Taught Programmes of Study

A screenshot showing the lengths of assessed coursework and exams as proposed by ASQC. The following indicates an appropriate and typical assessment load for a 20 credit module. The following relates to Assessed Coursework. A 20 Credit Module for 100% weighting should be under 4,000 words or equivalent. A 20 Credit Module for 70% weighting should be under 2,500 words or equivalent. A 20 Credit Module 50% weighting and under should be under 2,000 words or equivalent. For exams, the weighting is measured by length of exam rather than number of words. A 20 Credit Module exam 100% weighting should be under 3 hours. A 20 Credit Module 70% weighting should be under 2 hours. A 20 Credit Module exam 50% weighting and under should be under 1.5 hour.

Aim for one or summative assessment task per 10 credit module to reduce the burden of over-assessment on students and staff alike. This may increase to two summative assessment for 20 credits for example.
Consider synoptic assessment across modules to collaboratively meet programme learning outcomes (e.g. rather than each module assessing via exam consider one exam to assess knowledge and understanding across a range of modules)

Formative Feedback:

Ensure each summative assessment has a formative assessment component

Scaffolding:

Ensure there is scaffolding in assessment tasks For example, if assessing via groupwork at undergraduate level, design groupwork tasks across levels 4,5 and 6)

Authenticity:

Build in authenticity in assessment tasks (e.g. align assessments with real world experiences and engagements, placements etc).

Is it rocket science?

Holistic planning template

A decorative image showing a rocket ready to launch, to represent the toolkit’s example – Designing a BSc / MSc in Rocket Science. Coloured numbers remind us of the process diagram for Program Development stages and processes. The first is programme learning outcomes. Following on from here is Programme Assessments, Module Learning Outcomes, and Module Assessments and Learning Activities

In considering our MSc in Rocket Science example, we show below an extract for illustration purposes of how a programmatic assessment mapping exercise may look for a select number of PLOs.

Click here to view the programme assessment mapping.

In section 1, we covered:			In this section, we covered:	In the next section, we’ll fill this in:	In the fourth and final section, we’ll fill this in:
Learning outcomes category	Example Programme Learning Outcomes	Primary Graduate Attributes	Intended programmatic assessment methods	Potential Module Learning Outcomes	Potential Modular assessments	Potential Module Graduate Attributes addressed
Knowledge and Understanding	LO 1 - Understand and apply core theories, concepts, and techniques at the forefront of Rocket Science.	Independent and Critical Thinking	Exam Essay



	LO 2 - Apply basic and translational Rocket Science research in a range of professional settings, relating to human space flight.	Innovative, Enterprising and Commercial Awareness	Report



Intellectual Skills	LO 3 - Ability to select and apply appropriate mathematical techniques to solve problems in rocketry.	Independent and Critical Thinking	Exam Written coursework



	LO 4 Design and implement a range of experiments relating to the development of human space flight.	Ethical, Social and Environmental Awareness	Practical experiment



Practical and professional skills	LO 5 Work effectively within a range of interdisciplinary research teams.	Collaborative	Group Project



	LO 6 – Present scientific data relating to the construction of rockets in a clear and professional manner.	Effective Communicators	Presentation



Transferable skills	LO 7 Exercise initiative and personal responsibility within a range of professional contexts.	Reflection and resilience	Portfolio



	LO 8 Select and utilise appropriate research methods and resources in order to prepare a research project of your choosing.	Ethical, Social and Environmental Awareness	Research project

Help?

Wait! I still need more time to think about programmatic assessment!

Click here to explore our Deeper Dive section, which has a host of detail including more information on programmatic considerations and self-reflective prompts to guide you.

Where next?

The Process Diagram for Program Development stages and processes. The first is programme learning outcomes. Following on from here is Programme Assessments. The highlighted section is Module Learning Outcomes. Finally, there is Module Assessments and Learning Activities.

Understanding where and when assessments are best placed across the programme will help in informing the next stage of the four-stage process: the Module Learning Outcomes.

Deeper dive

Different interpretations of Programmatic Assessment

This section mentioned various interpretations of programmatic assessment. What are they?

The term programmatic assessment has many interpretations, varying by ‘levels’ of integration.

See for example this diagram from the PASS project where assessment strategies can range from low levels of programme integration and weightings to more high stakes assessment that fully integrates all programme learning outcomes:

A chart showing Different forms of Programme-Based Assessment. The X axis shows Weighting of the assessment in the final qualification from low to high. The Y axis shows the extent to which the assessment covers all the specified programme outcomes. Low weighted and low impact on assessment approaches are shown to be Typical assessment on single module and Some cross-module integrated assessment. Medium weighted and impact approaches are shown to be Integrative assessment across stages or levels and use of a Single Capstone module. High weighting and high impact approaches are shown to be Heavily weighted final integrative assessment and a single final integrative assessment.

More information on programmatic assessment and the PASS project is available in this video.

Programme development tip

It is worth remembering that innovative approaches to assessment may sit outside of the University's regulations. Explore your ideas in plenty of time with CLTA and the Quality and Standards Team.

Who should think programmatically?

Thinking programmatically should be embedded in all L&T activities, and all parties involved in the student journey should ideally think holistically in terms of assessment, from Programme Directors to individual Module Leaders.

When thinking about assessment strategies, innovations, changes, feedback approaches, assessment criteria and all things assessment and feedback - we should always think in the wider programmatic context. Examples of programmatic thinking are shown below.

As A Programme Director

A decorative image meant to represent a programme director talking with his programme team. He wears a blue blazer and stands with a wad of paper documents in his hands, smiling at his team of 5 staff who are gathered around a table in a circle.

Is there a broad range of assessments across the programme to sufficiently assess the programme learning outcomes?
Is the workload across the programme manageable and achievable for both staff and students?
Are there sufficient opportunities for formative assessment and feedback – within modules and across modules?
Does the programme support students in succeeding at modular assessment activities and provide assessment and feedback literacy so students understand the tasks?
Is there assessment bunching across the programme?
Are there opportunities for student choice in assessment across the programme?

As A Module Leader

Are my assessment activities in line with the programmatic learning outcomes and Cardiff graduate attributes?
Are assessments scaffolded throughout the programme, so students are prepared for my assessment?

How might Programmatic Assessment look?

Using our Rocket Science example, the below map shows a sample of how the programme may visualise their intended programmatic assessment types by level of study.

For an MSc you may use Periods 1,2,3 to denote 3 units or stages of the qualification, here in terms of visualising how the assessments may ‘look’ across the student journey we have used years 1,2,3 that may represent levels 4,5,6 for an undergraduate degree.

A map of a programme year by year, with formative and summative assessment mapped by feedback, type and interlinkages.

The full Word document is here, spanning several years and with formative and feedback elements added in as well as suggestions for more innovative programmatic assessment.

What are the benefits of Programmatic Assessment?

Prevents over-assessment and avoids assessment
Ensures alignment between programme and module level learning outcomes
Promotes diversity in assessment types
Encourages assessments which are accessible to all students and develop a range of skills
Allows learning activities to be co-ordinated across a programme (see the ABC approach to understanding learning activities on a programme)
Contributes to the development of graduate skills
Meet professional or regulatory requirements

Are there any disadvantages to Programmatic Assessment?

Programmatic approaches require collaboration, co-working and collegiality in order to ensure joined up thinking across what can be large complex and potentially interdisciplinary programmes. This can have time and resource pressures.
Programmatic approaches require data and information to ensure awareness of assessment types across a programme

Top Tips

Programme Directors may be able to share this information with module leaders; or Professional Services staff at your school may be able to help you in finding out about assessments on a programme by running reports in Business Objects.

What about Programmatic feedback?

Feedback should be embedded in assessment thinking and considered as ‘process’ not product (Carless, 2016). In this way Programmatic assessment includes consideration of programmatic feedback. Programmatic approaches help in deciding how and when students will create or receive feedback on a programme. A feedback strategy for the programme, should consider:

The number of opportunities that students receive formative and summative feedback on their programme;
When feedback is available and feedback activities undertaken;
How students can create, receive, and most importantly use feedback and what support mechanisms are available to ensure students can close the feedback loop.

Share your feedback

Next steps

You are on page 2 of 4 of the 4-stage process for Programme Development.
The next pages are:
3) Module Learning Outcomes
4) Module Assessment

❗ It is strongly recommended if you are designing a new programme or undergoing revalidation that you explore the following pages as you build up your Quality documentation. They are in a fixed order:

Or you could return to Programme Development and visit the other pages another time.