Using Multiple Evaluators in Assignments

The Multiple Evaluators feature in Assignments allows instructors to assign student assignment submissions to specific graders. This allows simple management of grading distribution to GAs/TAs for a large class, and additional settings enable the option for multiple graders to provide feedback before a final evaluation is published. Instructors can also prevent certain evaluators from being able to publish/release grades and feedback.

Table of Contents

• Set up the Assignment
  1. Select Evaluators
  2. Select who can publish the final evaluations to students
  3. Allocate students to specific evaluators
  4. Coordinate Multiple Evaluators
  5. Complete Set Up
• Evaluator Grading

Set up the Assignment

1. By default, all who have permission to grade (those with the role of Instructor, TA-Lead, TA-Grader or Departmental Support) are able to evaluate and publish all submissions. To manage evaluators and who can publish, begin by expanding the Evaluation & Feedback section when you are creating or editing the assignment.

Select Evaluators

2. Select Select Evaluators.
   Note: only those with permission to grade can be selected as an evaluator. This includes those with the role of Instructor, TA-Lead, TA-Grader or Departmental Support.

3. In the pop-up, select the individual(s) that you would like to be an evaluator for this assignment and click Done.

Note: Those with permission to grade (Instructor, TA-Lead, TA-Grader, Departmental Support) will always be able to see student submissions and override student grades in the Gradebook but if they are not selected as an evaluator, they will not be able to provide or change the assigned grades and feedback inside the Assignment tools for this assignment.

Select who can publish the final evaluations to students.

4. Under Publishers, in the drop-down menu, select one of:

• All evaluators can publish any evaluations
• Evaluators can only publish their own evaluations
• Manually select publishers. As the instructor, if you want to publish all the grades and feedback after you have reviewed the grades and feedback your TA/GAs have provided, select this option. Then click Select Publishers, select your name, and click Done.

Allocate students to specific evaluators

5. By default, all evaluators can grade all submissions. To allocate certain students to certain evaluators, click Manage Allocations.

6. You can manually allocate students to evaluators, or you can have Brightspace automatically allocate them.
   1. For automatic allocation, select from the drop-down list one of:
      • Allocate all evaluators to every learner
      • Randomly allocate 1 evaluator to every learner evenly

      (If you have selected more than 2 evaluators in step 3 above, you will see further options.)
      Then click Apply Allocation Method. Note: you can now make manual edits, if necessary.

   2. For manual allocation, add or remove check marks in the boxes that intersect an evaluator’s name and a student’s name.

7. When done managing allocations, click Done.

Coordinate Multiple Evaluators

8. If you have selected to allocate multiple evaluators to each student, choose how the evaluation process works. Select either:

• One shared evaluation
  All evaluators share the scoring and feedback area or work on the same evaluation together. Publishers release this evaluation to students.
• Multiple individual evaluations
  Evaluators work on their own evaluations separately. Publishers can see by a check mark beside evaluator’s names who has submitted their grading for aggregation. Publishers manually aggregate results into one final evaluation and release it to students.
  NOTE: If you select this option and have set up “Automatic Zero” in the Grades tool, publishers may not be able to aggregate results until after the due date.

Complete Set Up

9. Review the steps carefully to ensure you have completed the set up correctly for your use case. Finish creating or editing your assignment and click Save and Close.

Evaluator Grading

Those with grading permission who are not an evaluator for a particular assignment, will be able to see that particular assignment and the submissions, but will not be able to grade or provide feedback in the Assignment tool.

Those who are evaluators for an assignment will be able to see the assignment and all submissions, but will only be able to grade and provide feedback for those students who were allocated to them.

Evaluators can use the annotation tools, the rubric (if included), enter grades, and provide Overall Feedback.

When finished, if they have not been given the ability to Publish, they can Save Draft to continue reviewing the submission later, or, if your evaluation is complete, select Ready for Review or Ready to Aggregate. Evaluators who are able to Publish, will see Save Draft and Publish.

See the Evaluate assignments using the Assignments tool document, which also provides instructions for preparing multiple evaluations for publishing.

Tips for publishing and adding grades to the Gradebook are available in the Synchronize multi-evaluator grades between Grade Book and Assignments.

For questions, email Brightspace@uwindsor.ca, or open a ticket at www.uwindsor.ca/brightspace/ticket.