Developing Assessments

AI tools can help to develop assessments that are closely connected to learning outcomes, performance standards, and industry needs. Prior to using AI tools to help with any assessment development, ensure that institutional and departmental policy allows it. As well, instructors should consider consulting with educational developers and instructional designers at their institution to ensure that assessments are in line with institutional policies and practices.

Creating Questions

The Using AI to Enhance Learning chapter explores in detail how students can use AI tools to generate practice questions. In the same way, instructors can use AI tools to generate potential questions for assessments. Prompting AI with information about the course, program, level, and topics covered can yield a list of questions that may be reviewed and adapted for assessment. Another approach is to share a previous set of exam questions with an AI tool, and ask it to produce a new set of questions in order to vary assessments from term to term. Instructors should consult institutional and departmental policies before uploading internal information to an AI tool, and should consider uploading with data collection turned off.

AI tools can also be prompted to generate questions based on a supplied text. RRC Polytech Aerospace Manufacturing Technician instructor Chris Marek has engaged AI tools to create a bank of assessment questions by submitting a passage from an aviation manual and information about the course level and types of questions needed. Marek reported that questions produced were largely appropriate to the context and just needed reviewing and refining.^[1]

Instructors should consult copyright laws before uploading any protected content to an AI tool. Read more about AI tools and copyright considerations.

Alternate Assessment Formats

For instructors looking to diversify their assessment approach, AI tools can generate ideas for alternate assessment types or formats. As input, AI tools can accept an existing assessment or a course description and set of learning outcomes (if allowed by institutional and departmental policies). AI tools can also be prompted to produce assessment ideas related to a particular pedagogical approach, like problem-based or inquiry-based learning.^[2]

Scenarios, Case Studies, and Project Topics

AI tools can be used to create detailed and varied scenario-based assessments, and to present them in engaging and novel ways, such as a detailed letter from a client requesting services.^[3] Numerous RRC Polytech instructors have employed AI tools to generate a pool of possible group project topics on a given course and theme.^[4] In addition to basics about the course, level, and topic, sharing examples of previous project topics can lead to better output from an AI tool. Here is an example of a RRC Polytech School of Indigenous Education instructor prompting an AI tool for potential project topics. Click the image to view the full conversation (ChatGPT login required).

Even with a detailed prompt, AI tools will likely suggest some project topics that could work well, and others that miss the mark. One idea for reprompting is to evaluate the first set of project topics generated by an AI tool, let it know which ones are most useful and why, and ask for more that are similar.

Developing Rubrics

Creating rubrics is a time-consuming task for instructors and course developers. AI tools—both all-purpose tools and specialized ones like ClickUp—are well-suited to assist in the process of rubric development. If a set of rubric criteria are already established, an AI tool can be prompted with a course description, task description, and the list of criteria and levels of performance, and can produce a first draft of performance descriptions, which is often the most labour-intensive part of building a rubric.

Another approach is to supply an AI tool with a description of the course, the task and the learning outcomes that need to be addressed (if allowed by institutional and departmental policy), and the desired number of criteria and levels of performance. The AI tool can then be prompted to make suggestions for both criteria and descriptions. After AI tools have been reprompted for bulk editing, the rubric can be pasted into a Word document for further refining, or exported in a variety of file formats.

In the Winter 2024 term, one course in RRC Polytech’s Teaching for Learning in Applied Education program, a three-year program taken by instructors transitioning from industry to teaching roles, began allowing students to use AI tools to assist in a rubric-development assignment.^[5] Incorporating AI into rubric-creation assignments for new instructors ensures that they get an opportunity to experiment with AI tools and receive expert feedback on the suitability of their AI-assisted rubric.

Here is an example of a prompt used to generate a rubric. Click the image to read the full conversation (ChatGPT login required). By the end of the conversation, the rubric is still not ready for student use, but is at the point where manual refining will be more efficient than further reprompting.

Below is ChatGPT’s first attempt at a rubric, based on the supplied prompt. Click on the hotspots to view more information about some potential areas for reprompting and refinement. An accessible version of this rubric is also available.

Some issues with this rubric are well-suited to reprompting, such as asking ChatGPT to eliminate negative language in the rubric. Other issues may be more nuanced and may be quicker to refine manually, such as changing the adjectives in the header row.

For those who plan to develop multiple rubrics, adding some guidance to your custom instructions may be a time saver. An example custom instruction may read: Every time I ask you to create a rubric, it should have four performance levels listed in the top row. Those performance levels should read (from left to right): Developed (3 points), Developing (2 points), Beginning (1 point), Not yet (0 points). Please avoid negative language in rubric descriptions. My preferred adjective for the Developed category is consistently.

Rubric Image Description

The table below shows ChatGPT’s first attempt at generating a rubric for a mock job interview assignment. Following the rubric is a list of bullet points that outline some potential changes that instructional staff might like to make to the rubric before using it in their course.

Some Potential Issues with This Rubric

• In the first row, instructors may wish to use alternate adjectives that are more descriptive, avoid negative language, and are consistent with other course or department rubrics. Many rubrics list the adjectives first, with point values in parentheses.
• The Communication Skills criteria may be too broad, considering the rubric is for a communication course. Rubric developers may wish to get more specific or split it into multiple criteria.
• The Response to Common Interview Questions row mentions using the STAR model, but the associated learning outcome requires students to use a model such as STAR. Rubric descriptions need to indicate that STAR is not the only acceptable response model.
• The Context and Tone Adjustment criteria may be confusing. Students in this course will be aware of what it means to adjust tone but may not understand what it means to adjust context. Another word choice could increase clarity, like “adjusts tone to the context and to suit different questions and scenarios.”
• The rubric contains some negative language, like “it’s poorly executed or irrelevant.” Rubric developers might wish to choose more positive language, even when pointing out weaknesses in student performance.
• Unless a holistic category is desired, the Overall Impression and Professionalism criteria could be more specific. Since tone is already covered, as well as professionalism and the student’s communication skills, perhaps this category could be about non-verbal communication like body language, gestures, facial expressions, and eye contact.

Return to the Developing Rubrics Section