Rationales for the adaptations
So long as the Resume and Cover Letter assignment was a take-home assignment, I felt that authorizing the use of ChatGPT was the best option: some students would doubtless use it, and I did not want to create a situation where I would predictably have to sift ambiguous evidence to determine whether to recommend academic sanction based on probabilistic verdicts generated by ChatGPT detection tools.
Why not an in-class handwritten assignment?
In theory, I could have developed an in-class handwritten version of the assignment. However, I judged this option to be unfeasible. This is because both the resume and cover letter are time-consuming, partly due to all the norms governing those genres, partly since they must be customized to reflect a job posting and an individual’s background.
Also, students could not produce a realistically-formatted resume via handwriting.
Finally, it would have resulted in a poor simulation of and thus poor preparation for the time-unlimited typed at-home process students would use when preparing applications for real jobs.
Having authorized students to use ChatGPT, it was also logical to authorize other forms of support, such as receiving help from peers. While this more liberal approach to academic integrity left me uncomfortable, I believed the odds of students offloading the assignment were low for a number of reasons:
- The highly personal nature of the assignment and students’ belief in the real-world usefulness of the assignment would strongly motivate students to want to independently produce their resumes and cover letters to a great degree.
- The assignment is very complex, notably due to having to marry the conventions of 2 genres, a job posting, a student’s real background and my very specific grading criteria, such that arranging for a third party (e.g. ChatGPT) to creditably produce the assignment would require a great deal of time and effort.
- My experimentation with ChatGPT suggested to me that its tendency to produce vague, wordy, and impersonal writing would dissuade students from relying on it unthinkingly.
How ChatGPT may benefit students
What also helped offset my discomfort was my excitement at how ChatGPT might benefit my students.
First, ChatGPT was a potentially powerful tool to help students achieve the linguistically perfect resumes and cover letters demanded by employers: it generally produces correct English, and it has a considerable ability to identify and correct grammar errors and other issues with language. I, therefore, wanted my students to gain proficiency in using ChatGPT to help them create their applications for the course’s fictional jobs and then later for real jobs.
Second, using ChatGPT effectively for this assignment would require a knowledge of and reflection about resume and cover letter conventions in order to effectively design prompts and critically evaluate responses, potentially enhancing students’ learning.
Results and findings
In the end, the assignment average and standard deviation were appropriate for both sections. Therefore, allowing the use of ChatGPT did not result in high averages and low standard deviations as I had feared it might.
However, the average was somewhat low and the standard deviation was somewhat high compared to previous averages for this assignment. This grade data is consistent with my findings from reading students’ assignments: an unusually large number of cover letters featured some consistent problems. For example, the opening paragraphs of the cover letters were sometimes quite poorly executed and tended to be vague, while the cover letter body paragraphs were sometimes very unfocused.
These problems may have partly resulted from vague writing produced by ChatGPT. Regardless, I am certain that those problems resulted in large part from uncritical copying from sample cover letters I had supplied.
The post-assignment survey
After returning the assignment feedback and grades, I invited the students to complete a Google Forms survey regarding their experience with ChatGPT. I received 55 responses out of the 66 students registered in the courses. Here are some of the survey results:
- 47.3% of respondents reported that they opted to not use ChatGPT
- Of those who said they did use it, 73.3% mentioned that ChatGPT was only “somewhat helpful” or even “not at all helpful”
These results suggest that students’ reliance on and use of ChatGPT was limited.
The survey further reveals that those who did not use ChatGPT made that choice notably due to valuing personalized work, lacking time, ChatGPT being unavailable, or having doubts about its ability to meaningfully help.
Those who did use ChatGPT identified many ways in which it helped them:
- modifying and/or providing feedback about writing
- providing examples to inspire students’ own writing
- assisting with brainstorming
- checking and/or correcting grammar/syntax
- writing drafts of paragraphs/letters
- finding synonyms
- identifying a job posting’s keywords
- featuring a more professional tone
- writing with enhanced directness/precision/concision/variety
13 students explicitly declared that allowing the use of ChatGPT positively impacted the assignment, mostly because this resulted in greater realism; only 1 student explicitly declared that the assignment was negatively impacted.
Impacts of the use of ChatGPT
In general, the assignment was positively impacted by the integration of ChatGPT, notably due to enhanced relevance and enhanced learning:
- Students’ perceptions of the assignment’s relevance was increased.
- The assignment’s relevance was objectively increased by incorporating what is already a standard tool for resume/cover letter writing.
- Some students developed the skill of effectively using a large language model (LLM), specifically to prepare resumes/cover letters, partly by increasing their awareness of LLM strengths and limitations.
- The students who used ChatGPT likely increased their understanding of the resume and cover letter since crafting prompts and evaluating responses required further reflection about resume and cover letter conventions, and that reflection was from a novel perspective.
- To the extent that students inappropriately and unthinkingly copied example cover letters, lower grades fostered equity, and my assignment feedback made this copying an opportunity to foster learning (e.g. that cover letter openings should name specific qualifications).
- I spent no time or energy on investigating and prosecuting academic misconduct, and experienced none of the associated teacher-student conflicts that can compromise learning.
- I was empowered relative to my students and my credibility was increased by being the one teaching the students how to use ChatGPT and its limitations.
However, one main negative impact of allowing the use of ChatGPT must be considered: the more liberal approach to academic integrity resulted in a modestly increased number of students uncritically copying example cover letters and so reducing their learning at the composition stage of the assignment. This copying likely occurred because students knew they would not be charged with academic misconduct and may have perceived this copying to be less unethical or even sanctioned by me. Thus, I think it is highly likely that the unthinking copying of example cover letters was an unforeseen consequence of allowing students to use ChatGPT.
To try again?
Next year, I intend to proceed similarly with the Resume and Cover Letter assignment by allowing the use of ChatGPT and other forms of support. However, I will warn students against uncritically copying example resumes and cover letters, explaining how this will result in less effective applications and so lower grades.
Also, I will add the use cases of generating job posting keywords and generating synonyms to the “Using ChatGPT” section of the assignment instructions.
Finally, I’ll likely return the assignment’s weight in the course to 25% given that the assignment’s average and standard deviation have proven to be unproblematic despite allowing the use of ChatGPT.
Proceeding this way will be possible if ChatGPT limitations remain the same. However, if there are new publicly available versions of ChatGPT that are more capable, I will need to assess whether I can continue to allow students to use these new versions.
The Literary Essay assignment
I have recently repeated the exercise of allowing the use of ChatGPT with my Literary Survey class, this time with a take-home literary essay assignment on the medieval poem Sir Gawain and the Green Knight. Therefore, I modified my instructions by adapting the Academic Integrity section and by adding a Reporting on Support section.
