Managing Assessments

• Consider student effort hours when designing each assignment and use assessment equivalency guidance (see below) to design assessments (useful for consistency across modules and schools as well as when considering choice of mode of submission).
• Plan and monitor student assessment workload across each module, each semester and each programme.
• Use the assessment landscape to identify and limit assessment bunching.
• Use the assessment landscape to plan marking time and feedback release, ensuring the assessment burden for any member of staff is not excessive.
• Encourage colleagues to schedule marking time within their diaries.
• Plan marking calibration and moderation requirements have been considered within assessment timelines, so feedback release is not delayed.
• Communicate the marking and feedback process and timelines for feedback release clearly to students.
• Review the module and programme learning outcomes to see that learning outcomes are not being assessed in multiple situations.
• Consider using marking rubrics directly linked to grading criteria, or different marking techniques (e.g., video (e.g., video/audio feedback) which may be more efficient yet still effective for identified assessments.

When designing and setting assignments staff should have some concept of the notional time required by a student to complete the assignment.

Previous discussions have tended to focus on word limits as a measure of assessment load, however this is an inappropriate measure. For example, a 2000-word essay may be identified as being more work than a 500-word summary of a research article, however, the knowledge, understanding and student effort required to summarise a research article may be the same, if not greater, than the student effort required to write a 2000-word essay.

The following guidelines are therefore based on student effort as a measure of assessment volume (indicated by an estimation of the time required by a typical student to produce the work) enabling a more appropriate quantification of student workload and therefore enabling equivalence across modules.

When considering a 15 credit point module, the total hours associated with the module is 150hrs. The contact time for the module must be considered:

e.g.     Class contact = 44hrs

Independent study = 106 hrs

If the module is weighted 50% exam:50% coursework, then the independent study associated with each component should be divided between the two elements i.e., 53 hrs each.

For the coursework, we can consider that half the independent study time is related to reading and understanding leaving approximately 25 hrs devoted to directed coursework assignments.

For example, the module described above may have 2 assignments worth 20% each and a progress test worth 10%. The time guidance regarding student effort could be 10hrs for each of the assignments and 5 hrs for the test.

The following table has been produced to give examples of assessment loading.

Table 1: Assessment loading

The above table provides examples of assessment loads for 15 credit point modules. It is to be used as a guide to aid in the development of module assessment. There will obviously be debate as to whether one student would spend 10 hrs doing an assignment whereas another student may only spend 3 hrs and we recognise the diversity of the student body. However, we are trying to provide guidance to students to give them an indication of how long we think an assignment should take. Module teams should agree on appropriate time expectations for assignments which should then be stated within the assignment information on module sites.

e.g. “We would normally expect you to spend approximately 10hours on this assignment in addition to your regular independent study on this module”

In addition to summative assessment, formative assessment should be incorporated into the module and could be in the form of quizzes, progress tests, writing exercises. The time taken for formative assessments should be considered when determining the student effort for the whole assessment diet.

The marking of student work is important for two key points:

1. To identify the degree to which the student has met the identified learning outcomes/met the required standards; and
2. To provide feedback to the student so they understand the strengths and areas for development, in relation to the learning outcomes/required standards.

Marking requirements are identified with the University’s Policies and Regulations (UPR AS12).

Pseudonymous Marking

Unless it is unfeasible to do, we require all summative assessment to be marked without the marker knowing the identity of the student whose work they are marking.  This mitigates the risk of bias influencing the outcome of the marking process as well as the perception that it might have done so.  Therefore, students should only submit their work with a pseudonym simply so that if there are any administrative issues the work can be traced back to the relevant student.  Under our UPR AS12 a pseudonym includes the Student Record Number (SRN) of a student; therefore, students should normally be asked only to include their SRN on submitted work.

Instances where it will be unfeasible to mark without reference to the student's identity are:

• In-person assessments where the student can be identified during the assessment process;
• Assessments where the marker will already be familiar with individual submissions e.g. dissertations, project work, work with an individually negotiated title, work that builds on prior feedback;
• Objective tests such as online quizzes.

Marking and the Use of Generative AI (genAI)

Marking must always be conducted by a member of the module team and not by a genAI platform.

Submissions must not be uploaded into a genAI platform, even where misconduct is suspected unless there is prior written consent from the student in question.  Even where consent has been obtained the work must still not be uploaded if the student's work contains personal data belonging to other(s) or confidential information.  Where work has been uploaded inappropriately, it must be reported in line with the University's data protection policy.

Blind Double Marking

Blind double marking is where two markers work independently to determine a grade for a single piece of work, with neither marker being aware of the other’s decision when formulating their own grade.  Following a discussion of their individual grades, they will agree a single, final grade for the assignment. When an assessment is blind double marked, we do not require calibration and moderation is not required as the double marking and subsequent conversation provides the requisite assurance mechanism.

We would typically opt for blind double marking for dissertations and individual projects, but feel it can be applicable for all assignments at Levels 6 and 7, weighted 50% or more and where the question has been determined on an individual basis.

Calibration

Calibration is a required part of the marking process under UPR AS12 where an assignment is marked by more than one person whether it is a written, oral or artefact assignment (excluding assignments that are required to be blind, double-marked).  It involves each member of the marking team initially marking the same selected student submissions in order to establish an agreed benchmark of what a particular graded submission looks like and ensure agreement amongst the marking team as to how the grading criteria will be applied to the submissions.

Calibration aims to achieve consistent and fair grading decisions for all students.  The process relies on marking criteria tailored to the specific assignment, to guide discussions and ensure consistent application of assessment standards.

How should calibration be conducted?

We believe that good practice calibration involves the Module Leader selecting a minimum number of submissions for the marking team to review and these should be submissions that appear, at a glance, to be of variable quality.  When the assessment is an exam, the selection should include a spread of the questions answered by students.  (Subject to the number of submissions, five is usually a good starting number).

All members of the marking team should then meet (virtually or face-to-face) within days of the assignment deadline to review the selected submissions and the proposed marks of each individual marker against the background of the assessment brief, learning outcomes and grading criteria.  When meeting, markers should discuss the strengths and weaknesses of each piece of work, highlighting key features that correspond to specific grade boundaries.  The marking team will then agree a final mark for each submission to act as benchmarks throughout the marking process.

Discussions should also be had, and agreement reached, in the meeting as to what markers will be looking for in terms of each grade boundary, drawing on the experience of the collectively marked submissions. For example, agreement about what degree of complexity of expression or interpretation is required within each of the grade boundaries.

Marking teams should engage in ongoing communication through the marking process to raise further issues and queries, as they are identified, that may result in markers taking an approach to grading out of line with the rest of the team.

Moderation

Moderation is distinct from marking and calibration and is required at Herts for all summative assessments (with two exceptions, see below).  Its purpose is to ensure that proper processes have been followed and that the quality of marking and feedback is at the level it should be and that fairness and consistency are evident.  Moderation is not about swapping a proposed grade for one that the moderator thinks a student should have received.

Because of the important role that effective moderation plays in helping students feel assured about the fairness of their marks, programmes should provide information to students on the marking process, including calibration, moderation, and the role of External Examiners.

Moderation is not required for assessments that are subject to the requirements of blind double marking nor for assessments that objectively marked using a computer.  However, in the latter situation it is still best practice to review all the grades and moderate questions with a high number of incorrect answers.

Moderator Eligibility

The moderator should be identified prior to the marking starting and it is best practice that they have been involved in discussions regarding the design of the assessment and they are aware of what information has been, or is being, supplied to students.

Unless the relevant Dean approves otherwise, a moderator must:

• Have two years' experience working in UK HE;
• Not be someone who has been involved in the marking process.

Approval from the Dean should only be given in exceptional circumstances.

The Moderation Process and Requirements

It is the responsibility of the Module Leader to ensure that moderation has taken place prior to the grades being released to students; however, not all of the marking needs to have been completed before moderation occurs.  It is best practice to commence the moderation earlier in the marking window, e.g. after 10 days or when a sufficient number of submissions have been marked to enable the minimum amount of moderation to occur. This has two advantages:

1. It reduces the time pressure on the moderator and ensures that they have sufficient time to review the necessary number of submissions in depth.
2. It allows groups of markers to know early whether they are deviating from the other markers and recalibrate their grades before they have all been marked.  (The moderator may need to check again once all submissions have been marked in case there are any grade boundaries that were not considered previously).

The outcome of the moderation process and the identification of the sampled submissions must be recorded within the module's Virtual Learning Environment.

Minimum sample requirements

• The sample of moderated scripts should be equal in number to the square root of the total number of submissions, as a minimum.
• Where there is more than one marker, a minimum of three marked submissions from each marker should be in the sample.
• All grade boundaries awarded during the marking process should be included in the sample.
  • Fails grades should be included in the moderation sample as a reflection of the different grade boundaries, but not every fail grade needs to be moderated.
• There must never be fewer than five submissions in the sample unless the total number of submissions is fewer than five in which case all submissions should form the sample.

Therefore, in most instances a number of submissions that is greater in number than the square root will be required to be moderated to ensure that all grade ranges from all markers are included in the moderation sample.

In-module retrieval (IMR) is a reassessment opportunity given to a student that has failed an assessment initially and provides them with an attempt to retake the initial assessment and pass it ahead of the formal referral opportunity given by exam board.   IMR is not an opportunity for students to improve on a pass grade.

We believe that there is a straight-forward pedagogic argument for IMR; a retrieval can be closer to the time of the original learning than a referral, and it encourages students to learn from their mistakes and to use feedback productively.  Further, IMR can, in principle, help to maximise student opportunities for progression when it is possible for another attempt to be made at a coursework, practical or test prior to the exam board.  IMR particularly has potential when failure on an individual assessment triggers an automatic referral because it assesses unique learning outcome/s. If IMR is used here, then it may be possible to avoid a referral altogether. In-module retrieval could also play an important part in improving first-sit pass rates, therefore contributing to the improvement of continuation rates and reduction in the awarding gap. Where skills are gradually developed, the need to promptly assess and pass before moving on in terms of sequential learning is important for example in the case of technical or practical skills. IMR also supports our adoption of more authentic assessments as there will be a move away from assessment product to assessment process, with IMR using the feedback and encouraging students to be more reflective. We also believe that providing timely feedback should encourage students to act on it in a timely fashion.

While we have not mandated the use of IMR, our Schools have been given an opportunity to consider where it would be appropriate to introduce.

Prior to introducing IMR it is important for programme teams to consider:

• When might IMR be positively encourage or permitted?
• When might it not be appropriate to offer?
• What are the assurance mechanisms needed to implement appropriately?
• How would information be communicated to students?

Under our approach to IMR, it is important to be clear that it is not a formal referral as only an Exam Board can grant a referral opportunity. However, if an IMR opportunity is not successful, then it progresses to referral. We do not consider the retrieval opportunity as an attempt or sitting for the purposes of calculating how many attempts a student has had within an academic year.

Circumstances when it is appropriate to offer

We believe that IMR should only be offered for first sit and deferral assessments, but not on referral assessments.  Before deciding to offer IMR the Module Leader and Programme Leader must be satisfied that it will be possible to schedule and mark the retrieval assessment in a timely manner, in line with university regulations, and no later than four weeks from the end of the module.  Consideration as to whether it is practicable to offer IMR within a module must be given for each instance that the module runs within the academic year.  IMR should only be offered within module instances where it is appropriate to do so; therefore, it is permissible to offer IMR in for one module instance even if for another instance it would be logistically inappropriate to offer IMR.

Examples of when it may be appropriate to offer IMR include assessments that are:

• Competency based;
• Marked as 'pass'/'fail;
• Part of a sequence where the students ability to undertake the next assessment will be disrupted by having to wait for a referral;
• Practical and that are marked on the day;
• Tests, particularly those where the questions are drawn from question banks within the VLE;

Assessments when it would be inappropriate to offer

We would consider it inappropriate to offer IMR where:

• The original assessment is due in the last 4 weeks of the module except where marking and feedback is provided immediately;
• The assessment design is an exam or is otherwise impracticable to repeat e.g. group assessments;
• It is prevented by a PSRB that oversees the programme in question;
• The student has not made a first attempt;
• The student is under investigation for academic misconduct in relation to the assessment in question.

Practical considerations when offering in-module retrieval

Advice should be provided to students regarding whether they should take up any IMR opportunity, especially where a student has more than one failed assignment in the same period or has other first sit assignments at the same time.  It will not always be in the student’s best interests to undertake the IMR opportunity.

To further guard against academic misconduct, it is advisable to ask students to include a short self-reflection on how the feedback from the original submission was acted on.

Where a retrieval assessment is replicating the original assessment, it should not require review prior to their release as this will have been undertaken previously; however, where the retrieval assessment is different to the original then it is important that the usual academic scrutiny processes are undertaken.  In any event, the retrieval submissions will need calibrating and moderating as usual.

Communication to students

The programme site within the VLE should set out the programme’s approach to in-module retrieval and make clear the extent that it is available to students so that it is clear and transparent from the start of their course.  This information should include informing students that it may not be available on all modules or all assessments within a module.  To maintain student confidence in the approach, it is important to explain to students that different practices are not necessarily inconsistent or unfair.  This is why it is important to have a clear rationale setting out why IMR is allowed and when (and similarly when it is not).  The information should include academic advice for students to support their decision making as to whether an opportunity for IMR should be taken up or not and who they can talk to further about their decision.

Within module sites on the VLE, information specific to that module should also be provided to students setting out when IMR is being offered and why (or why it is not offered). This should include the dates of the retrieval opportunity, ensuring that they align with any requirements for feedback turnaround times and deadlines for submitting marks to the relevant Exam Board.  However, we consider that the availability of IMR should not be required on module DMDs and that best practice is not to include it so that maximum flexibility regarding assessment design is maintained.

Every assessment brief must also state clearly whether the assessment carries an IMR opportunity. If available, it should inform student whether there is the possibility of being granted an extension or exceptional circumstances.  Our position is that these are not available for IMR opportunities.