Survey on AI Use

As I say in the course's AI policy, I am not necessarily against all AI on principle, and think that there may well be a place for it: "I dream of a day in which we have a reliable and effective AI to which we can consign the tedious, semi-intellectual bureaucratic labor in which we see no inherent value: filing taxes, writing reports, filling out forms, university admin, and so on." I therefore thought that the task of summarizing such a diverse range of comments might be something that ChatGPT could do, with perhaps helpful results. So here is what happened when I asked it to summarize and respond to your comments...

OK, I did ask ChatGPT to summarize and respond to your answers... but (ironically!) it didn't do a very good job. In fact, I asked it twice (and for full disclosure and transparency, take a look at my prompts and its response), but it struck me that, though its summary was plausible (because ChatGPT is always plausible... AI works to produce answers that look as though they could be legitimate), it also hallucinated or made stuff up.

To take one fairly clear example, ChatGPT included in its summary the following:

"Students express skepticism about:
* the ability to detect AI use
* fairness of enforcement
* consistency across students"

Now, it's quite possible (again, quite plausible) that you may be skeptical about such things... but in fact, none of your responses mentioned any of these three issues.

Similarly, ChatGPT also included in its summary the following:

"Students repeatedly ask for:
* examples of acceptable use
* assignments that incorporate AI
* discussion of its strengths and limits"

Again, though these are (in my view) in principle entirely reasonable things to ask for, as a matter of fact none of your responses asked for them, let alone doing so "repeatedly."

In short, and again ironically (or symptomatically) ChatGPT produced a general, somewhat banal, plausible, but fundamentally inaccurate picture of this survey.

Or in other words, it gave me a summary of what it thought you might say, or what it could be reasonable or likely for you to say, but ignored what in fact you did say.

(There's a political lesson here, too, I think... what happens to democracy when AI is used to consult people, but ignores what they actually say and think in favour of its notion of what they "should" be thinking.)

To be honest, I'm surprised... I thought it would have done better. But here we are.

Update: I was talking about ChatGPT's failures with some friends, and one of them put in the same query to Claude, adding the following qualification to the original prompt: "Create a summary of the attached survey. It is vital you do not make inferences not supported by the responses in the survey. Be honest and thorough." The result was much better, and I'm pasting it in below. This is not to say that Claude is necessarily "better" than ChatGPT. There are various possibilities as to why ChatGPT failed this task so badly. The main point is, however, that we are still (I think) far from a position in which we can trust AI. (And even when or if we get to that position, other questions are still pending.)

Summary: Survey on AI Use — RMST 202 (Literatures and Cultures of the Romance World II), UBC, Week 10, 2026

Context and Purpose

The survey was conducted in week ten of the 2026 term in RMST 202, a Romance Studies course at UBC. The instructors stated they were interested in the impact of generative AI on education, particularly in Arts, and in the relationship between AI use and the course's contract grading system. They expressed a belief that contract grading depends on trust and a concern that AI use could degrade that trust, while also suggesting contract grading might reduce student anxiety and thus reduce motivation to use AI. Students were asked to respond anonymously.

Section A: AI Policy in RMST 202

Q1. Is the course AI policy clear?

All 59 respondents answered yes. No one answered no. Comments indicated students understood the policy as a straightforward prohibition on generative AI. Some noted it was explained from the first day of class or available on the course website. One student noted they hadn't actually read the policy but assumed it was clear.

Q2. Have the reasons for the AI policy been sufficiently explained?

56 respondents said yes; no one said no. A handful of comments indicated students found the rationale self-evident in a literature course context. One student said they hadn't read the reasons but the policy itself made sense. One student suggested the reasons could be reinforced more throughout the term, particularly around assessment periods. One student expressed strong personal agreement, citing concerns about AI diminishing critical thinking.

Q3. Is the course AI policy fair?

56 respondents said yes; no one said no. Several students connected the fairness of the policy to the contract grading system, noting that the flexible grading removed the need for AI as a crutch. One student said they wished limited AI use were permitted to help articulate ideas. Another said they thought "a little bit should be allowed." One student said they hadn't read the policy.

Q4. If the AI policy were changed to permit use, could this help your learning?

8 respondents said yes; 34 said no. This is the most substantive question in terms of qualitative variation. Key themes in the responses:

• Many students said AI would make them lazy or reduce engagement with the texts. Several made this point explicitly and more than once.
• Multiple students noted that contract grading already removes grade-related pressure, making AI assistance unnecessary or redundant.
• Some students said the course's reading-and-reflection blog format simply doesn't require AI.
• A smaller number said AI might help in limited ways — brainstorming, organizing thoughts, clarifying complex vocabulary, or honing writing — but most of these respondents still expressed ambivalence or qualified their yes answers.
• Some respondents expressed concern about AI accuracy and reliability.
• One student noted that not using AI had actually helped their critical thinking and learning.
• One student said AI use might be a "slippery slope."
• One student said it wouldn't affect them because they don't use AI.

Section B: AI Use in Other Courses

Q5. How often have you been permitted or encouraged to use AI tools?

Responses were varied and did not follow a simple yes/no format. Key patterns:

• Many students said AI use is rarely or never permitted in their Arts courses.
• Several noted AI is more common or more accepted in non-Arts disciplines — Economics, Computer Science, COGS (Cognitive Science), and Sauder (Business) were mentioned specifically.
• When permitted, it was usually for limited purposes: brainstorming, generating practice questions, summarizing content for study purposes, grammar/editing help, or finding sources — rarely for producing submitted work.
• One student mentioned a Biology course that required AI use to compare search engines (Google vs. AI). They described it as "stupid."
• One student noted a poetry course where ChatGPT was encouraged to find poems quickly, but submitted work could not be AI-generated.
• A few students reported that approximately 50% of their courses allowed AI in some form.
• One student mentioned a professor who uses AI to write Canvas announcements.

Q6. Have you found permitted or encouraged AI use to be helpful?

Responses were mixed. Key patterns:

• Many students said no, or expressed reservations about AI making them less engaged, lazier, or more confused.
• Students who found it helpful tended to cite specific, limited uses: studying for exams, brainstorming, understanding concepts, organizing thoughts, generating practice questions, or finding sources.
• Several students said they chose not to use AI even when permitted.
• One student expressed concern about "popcorn brain."
• One student noted that in group projects, classmates using AI created more work for others who had to correct errors.
• Students in Economics and Sauder tended to report more positive experiences with AI than Arts students.
• One student said helpfulness depended on whether the course required opinions or facts, suggesting they found AI more useful in fact-based courses.

Q7. Have you ever broken or bent the rules with non-permitted or unauthorized AI use?

The majority of respondents said no. Those who said yes or qualified their answers described:

• Use in high school, before widespread AI policies existed.
• Summarizing research papers or long slides.
• Using AI to check whether they had met assignment requirements.
• Using the AI preview section in Google search results, mainly for STEM courses.
• Using AI as a study tool (generating MCQ questions, study guides, explaining concepts) without using it to produce submitted work.
• One student said "accidentally."
• One student said they lacked "the guts to" use it unauthorized.
• One student said they couldn't risk a zero.

Q8. Have you found non-permitted or unauthorized AI use to be helpful?

Most respondents said no or expressed ambivalence. Key themes:

• Several students said unauthorized use encouraged laziness, procrastination, or inaccuracy.
• A few said it was helpful for specific tasks: summarizing papers, creating study guides, getting quick reminders on material, or checking citations.
• Some students said it was not helpful for learning in the long run, even if useful in the short term.
• One student said they were "too scared to risk their degree."
• Several students answered N/A, indicating they had never used AI in a non-permitted context.

Instructor's Note on ChatGPT Summarization

The survey post includes an account of the instructor asking ChatGPT to summarize the student responses. The instructor reports that ChatGPT produced a plausible-sounding but inaccurate summary, including claims that students had raised concerns about AI detection fairness and enforcement consistency, and that students had "repeatedly" asked for examples of acceptable use and assignments incorporating AI — none of which appeared in the actual responses. The instructor used this as an illustration of AI hallucination, noting that ChatGPT generated a summary of what students might plausibly say rather than what they actually said. This was the direct motivation for the instructor writing their own manual summary, which the survey page also contains (the above is drawn from the raw student responses, not the instructor's summary).

Overall Observations (Drawn Directly from Responses)

• Near-unanimous agreement that the AI policy is clear, explained, and fair.
• Strong majority rejection of the idea that permitting AI would help their learning in this course.
• Contract grading was frequently cited as a reason AI feels unnecessary.
• AI use in other courses is sporadic and more common outside Arts.
• The majority report not having used AI in unauthorized ways; those who did described limited, task-specific use.
• Responses about unauthorized use were largely negative or ambivalent in terms of perceived helpfulness.