ChatGPT is an Artificial Intelligence (AI) chatbot released in late 2022. ChatGPT is a large language model which can produce language that sounds like a human. It has learned the meanings of words and their contexts based a vast amount of inputted text. What makes ChatGPT notable is the degree of its sophistication in its responses. The machine operates by selecting the best word or phrase most likely to conform to regular convention based on language context. ChatGPT does not “search” the internet (the current version was trained on material up to the end of 2021) nor does it “compute” mathematical queries. However, Microsoft is in the process of releasing their search engine “Bing” which has been integrated with ChatGPT allowing it to search the internet.

While ChatGPT is quite accurate in many cases, if it does not know the answer it will often “hallucinate” and write a convincing-sounding response that is either partially or entirely inaccurate. This fake information can be anything from a peer reviewed source, a historical figure, events and so on. This approach extends to the inclusion of citations, which will follow convention, but may—or may not—be real.

Developers of OpenAI also note the presence of bias within the system, resulting from its approach to draw upon language conventions that reflect systemic inequities and power-imbalances.

One poll of students provides some indication of how widely ChatGPT is used on campus. Most respondents (83%) did not use ChatGPT for their final assignments and exams in Fall 2022 but of those who did, the majority used it for “brainstorming, outlining and forming ideas” (60%), while less than 15% said they “submitted written material from ChatGPT” with or without edits.

ChatGPT will perform well in responding to certain assessment types as follows:

1. Multiple choice, the AI will typically predict the most likely to be correct answer (reported to typically achieve a B grade).

2. Open-ended questions which ask students about facts or comprehension (e.g., summarize, explain) or even some forms of critical thinking (e.g., analyze, critique, compare).

3. Creative assignments as ChatGPT can create novel examples to illustrate concepts, use concepts to analyze media examples, give convincing personal reflections to podcast transcripts, make educated guesses about how concepts would function in hypothetical worlds, and be creative in its responses (e.g., creating a response in the form of a "PSA", a social media post, a letter, or a script for a lecture).

Two areas to focus on are the general culture around ChatGPT and its use, and specific options for creating assessments that could lead to greater academic integrity.

Culture around ChatGPT

• Explicitly address ChatGPT in class so that students are aware of its strengths and limitations as well as what is acceptable vs. unacceptable use of ChatGPT in your course, include a summary statement in your syllabus.

• Adopt an empathic or less punitive responses to late submissions (i.e., if students know they can have extra time they may be less likely to “rush” their work by using ChatGPT).

• Ask students to explain orally what they have written if their work seems uncharacteristic or suspect.

• Integrate the use of ChatGPT into assignments so that its use is encouraged under certain circumstances and students can benefit from what it offers. Ask students to share the language prepared by ChatGPT alongside their final draft and critically comment on the limits of AI or how they revised and refined the text.

• Pose questions that encourage metacognition or reflection through exit tickets or cognitive wrappers.

• Run your own assessments through ChatGPT, perhaps several times, to be aware of how it answers them. Use this insight to adjust your approach to assessment and/or grading rubric or be able to detect patterns or similarities whilst reviewing student work.

Assessment Design

• Replace take-home tests with in-person assessments (e.g, in-class quizzes, in-person midterms or final exams).

• Make sure written assignments include higher-level critical thinking (i.e., not simply recall, basic comprehension, or synthesis).

• Have multi-part assessments (e.g., drafts of papers, stages of creating the final product).

• Have oral assessments or presentations (note that ChatGPT can produce “scripts” for presentations).

• Ask students to critique what AI-generated content (e.g., having students’ fact-check AI generated content, add context/depth/meaning to what was generated, add citations based on course material).

• Ask questions that require students to respond using content after 2021 which is when ChatGPT’s training ended (e.g., analyzing a movie, discussing a YouTube clip).

• Ask questions that require students to respond using material specific to your course (e.g., citing lecture slides in their response, discussing in-class material, citing the textbook or course readings).

• Have “authentic” assessments that students are more invested in and less likely to cheat on (e.g., giving students more choice over how they are assessed, giving students more choices about what to research/write about), asking students to draw on personal experiences or strengths.

• Establish common intellectual experiences which both establishes a culture of acceptance around questions or areas of confusion (muddiest point) and students are expected to publicly demonstrate their competence.

Many of the strategies proposed above (e.g, oral exams, projects with multi-step components) may not be feasible in large classes, for those without sufficient/competent support from teaching assistants and/or for part-time faculty or others who do not have as much time.

Designing new assessments is time-consuming, especially for part-time faculty or other lower paid or marginalized faculty.

In-person assessments are typically less accessible and inclusive than those which are online.

Not all students have the same ability to engage in higher order critical thinking so proper scaffolding and explicit teaching of these skills may be necessary before assessing students on these skills.

Not all students will have access to the pro versions of these large language models, which could lead to educational disparity

It is important to avoid setting up an adversarial relationship between students and faculty where faculty act as though they assume students are cheating or acting in bad faith.

It is important to avoid getting into an arms race between students using AI and faculty trying to detect the use of AI. There are AI detectors but they are not perfect and it is relatively easy to fool them by adding in grammar/spelling mistakes to AI generated text in order to mimic how a human would write.

Implicit bias present within the stores of information ChatGPT draws upon and potentially in how the AI was trained.

AI is constantly improving and becoming more accessible, and strategies which work today may not work in the (near) future as AI is integrated into more applications (e.g., Bing).

In responding to this new technology faculty may seek new approaches to assessment which ultimately develop students’ higher order thinking skills.

Will the introduction of more stringent assessment conditions negatively impact students with learning difficulties, accessibility concerns and/or other barriers to access?

Will this produce strata of cohorts divided between those who perform better than AI and those who perform below AI?

Additional links to resources on ChatGPT can be found here:

1. CriticalAI.org

2. University of Lethbridge

3. Video Introduction