The Complexities of Process Tracking

Experimenting with a new accountability measure for the age of AI

Photo by Tobias Tullius on Unsplash

In my last post, I reflected on the larger problems we’re facing around AI misuse in college classrooms right now. As I noted, I think the issue is structural but that there are many things instructors can do in their individual classrooms to motivate student learning and, when needed, to build in accountability measures that ensure students are doing their own work.

One accountability measure I mentioned, and that others have written about, is process tracking. Basically, process tracking involves monitoring the process by which students craft their assignments. One way to do this, for example, might be to dive into the version history of a Google document, viewing the timestamps of the essay, how long it took to put together, the order in which the words were written, the erasures and false starts, what words were copied and pasted from other sources, etc. A somewhat newer tool for process tracking is Grammarly Authorship, which actually records a student’s screen as they write and generates a report on their “originality.”

We talked about this tool, and others, at my university’s AI Institute this week. It got me thinking a lot about the potential advantages and disadvantages of process tracking.

On the one hand, as

Anna Mills

points out, this kind of accountability measure is better than AI detection, which is notoriously unreliable and easy for students to circumvent. I also like it better than using in-person proctors or digital proctoring software, since those involve not only surveillance but also, typically, time pressure. It’s difficult for students to devote sufficient cognitive energy to their work if they’re hearing a ticking clock in their heads and feeling eyes on their back the whole time.

Beyond being a way to determine whether or not students are doing their own work, I also see the appeal in process tracking for supporting learning. A lot of what I’m trying to teach first-year students is how to engage in the writing process effectively. They can’t learn about their own process without being aware of and reflecting on it, something that process tracking could help facilitate. Theoretically, if I, as the instructor, also have an inside view of their writing process, I could help them do a better job of examining and enhancing it.

On the other hand, as my colleague

Marc Watkins

points out, being continuously surveilled might have some negative downstream effects for developing writers. At best, it’s an invasion of privacy. Of his own experience, Marc writes,

If an editor saw my writing process they’d think I was mad. If I was a student and a writing instructor saw how I write, with time stamps, keystroke entries, what was copied, what was deleted, and yes, what was made by AI, what would that person think of me? I feel entirely naked just describing how I write—I would be mortified if someone could see that entire process.

In general, I’m anti-surveillance, so when I read Marc’s piece, I was inclined to agree, at least on an intellectual level. It wasn’t until I began writing this post that I started to understand the sentiment on a deeper, emotional level.

---

My new misgivings are the result of an experiment I’m trying this week. I’m screen-recording my writing process for this post and publishing a video of it along with the text. Here’s what I ended up with:

I honestly didn’t think that making my writing process visible like this would bother me. Unlike Marc, I don’t feel particularly uneasy about sharing this kind of thing. But all that changed when I actually started writing. Here are a few observations about what it’s like to draft under surveillance:

One potentially good thing about recording my process is that it’s keeping me focused. I don’t want to check my email or answer a text or doomscroll on Bluesky or start doing another task because either 1) people would watch me do it, which would be embarrassing, or 2) I’d have to go through the trouble of stopping and restarting the recording several times, which turns out to be pretty inconvenient. I also don’t want to stare into space too long, lest those watching the recording think I’m…I don’t know—too slow or something. How, exactly, this enhanced focus is affecting my writing process, or the text I ultimately produce, remains to be seen.

I’m also encountering a general fear of judgement about what and how I write. What if people think my process is weird or wrong? What if I type something that I change my mind about later and, in retrospect, don’t want people to see? I mean, that’s why we delete things, right? Because we don’t want people to read them? What if I vacillate between ending the previous sentence with a question mark and an exclamation point several times and all this waffling is out there in public for people to see?

Troublingly, screen recording with the expectation of an audience for my process is changing the way that I write. I’m less inclined to navigate away from this page to explore other sources and ideas or pause too long to think about what I’m typing. I’m writing a bit faster than I normally do. I’ll probably end up writing this in 4-5 longer, focused sessions rather than in the 10-12 micro sessions I usually take to draft a post. It’s possible (though this is only on the edge of my consciousness right now) that I’m taking fewer risks as I’m writing. I certainly feel more vulnerable, which probably makes it less likely that I will type anything that’s really out there, whether or not it ends up in the final cut. I’m having difficulty focusing on what I’m writing because half of my attention is focused on the performance of writing.

Worst of all, after taking writing breaks, I kind of dread coming back to this draft and am anxious to be done with it. This never happens to me under ordinary circumstances, at least not for blog posts.

I genuinely didn’t expect to have these kinds of feelings about recording my writing process. If all this fear of judgement is popping up for me, an experienced writer, imagine how a developing writer must feel. If I’m anxious about what people will think of my writing process when there are no stakes attached to their opinions, imagine what it must be like for a student who is being graded on their work. If a person like me, with an almost lifelong affinity for writing, is dreading the activity when it’s being recorded, imagine how much more students will dread it.

---

All this reminds me of something my colleague

Sarah Silverman

said about AI process tracking on LinkedIn recently:

I do think teaching strategies related to AI use that ask students to make their writing process legible to instructors (as a form of transparency and even metacognition) would have posed a barrier to me (as a neurodivergent student). I am very very reluctant to describe my process of writing or doing pretty much any kind of work to someone else if they are going to be evaluating the process in addition to the product. This is because I have had so many experiences of being told that my process is wrong, and failing when I tried to implement someone else’s “correct” process. I would write a totally different essay if I knew I was going to have to share my process, and it would be much harder for me.

As someone with a pretty neurotypical brain, this wouldn’t have occurred to me. But even as a neurotypical person, I’m experiencing something like what Sarah describes in that I’m writing differently because I know the process will be shared, and as a result, I’m having a slightly more difficult time of it. Moreover, now that I’m thinking about it more deeply, I do remember a few instances of students being uneasy about sharing their writing processes when the subject has come up in my class.

Last fall, for instance, I got this stellar essay from a first-year student in the first two weeks of class. It was supposed to be a low-stakes, informal, diagnostic assignment, and it was clear the student had gone above and beyond the call of duty. I got kind of curious about how much time they had spent on the essay, so I opened up the version history of the document to look. I saw that they had been working on it for many more hours than necessary and had even stayed up until 4:00 am to submit the draft on the day it was due.

This was pretty concerning to me: I don’t want students to sacrifice sleep to complete essays for my class (I’ll happily give them another few days), and I certainly don’t want them to stay up until sunrise putting in hours of work for an informal diagnostic essay! So, in my comments, I noted how important it was to prioritize tasks and develop healthy working habits in college—and that if they ever found themself putting assignments for my class before sleep, they should come and talk to me.

I didn’t think much more about it until we had a conversation later in the semester and they mentioned how embarrassed they were that I had seen those timestamps. The fact that I had seen them led to some good conversations about their writing habits and struggles with perfectionism. But I hadn’t realized just how sensitive this kind of information could be for students. It did feel like I had invaded the student’s privacy by locating, and mentioning, information about when they were writing their essays.

I suspect that many students are—like Marc, Sarah, the student above, and now myself—uncomfortable with making their writing processes visible. But some are also turning to self-surveillance as a way of protecting themselves from allegations of AI misuse. A piece published last week in the New York Times tells the story of Leigh Burrell, a student at the University of Houston-Downtown who was falsely accused of using AI to write one of her assignments. She was able to prove her innocence, but only after a lot of effort (not to mention anxiety). After that incident, Leigh began recording her screen as she drafted essays, saving those recordings in case she was falsely accused of AI misuse in other classes.

Another student, a high school senior named Sydney Gill, noted that she had “second-guessed her writing ever since an essay she entered in a writing competition in late 2023 was wrongly marked as A.I.-generated.” The experience had a lasting effect on Sydney: “I don’t want to say it’s life-changing, but it definitely altered how I approach all of my writing in the future,” she said.

I’d love to hear more from these students about how the current landscape is changing their writing processes—and just as importantly, their relationships with school. I can imagine that it’s hard to learn when you’re spending a lot of your cognitive energy worrying about whether or not someone will accuse you of dishonesty. At the very least, you’re probably more focused on making your work appear simultaneously human-crafted and worthy of a high grade than you are on actually learning something.

---

So, what are we, as instructors, supposed to do? Despite its flaws, process tracking seems like one of the most learning-focused forms of accountability we have. Admittedly, I have gotten some good mileage out of checking up on document version histories. When something feels off about a sentence or paragraph in a student essay and the version history indicates that it has been copied and pasted, I ask students about it: “Where did this come from? Do you know what it means? Why did you put it in? What made you think this idea was better than yours? What made you think it was better expressed than something you could have written?” These are almost always extremely productive conversations, and students and I both walk away from them having learned a lot.

So, being able to track and discuss students’ processes can be really good! But as I noted above, it also feels invasive in many cases. And it may encourage students to focus on curating a writing process for the eyes of their instructor rather than exploring and creating a writing process for themselves.

I think the first thing I would recommend to fellow instructors is this: if you’re considering using Grammarly Authorship or other forms of screen recording in your classes, try recording yourself for a while first, with an actual or imagined audience in mind. See how it feels to you and what kind of issues you encounter. Maybe even write a reflective blog post on your experience. It might provide you with a lot of insight. I will say that before writing this post, I considered asking students to screen record their process for writing their major essays—not necessarily for my use as the instructor but so they could revisit and examine their own writing habits. I am now reconsidering this idea.

I do think, however, if we want to monitor students’ writing processes, we should move away from the idea of “tracking” and focus instead on “reflection.” I don’t need to track students’ keystrokes or comb through videos and version histories to make sure they aren’t cheating. But I, and students, do need a mechanism to examine their writing processes, both to help them develop some self-knowledge as writers and to ensure that they’re doing their own thinking rather than over-relying on AI. If a student doesn’t have thoughts on their process and isn’t able to describe it to me, we have a problem—not because they’re cheating but because they aren’t learning.

Here’s what I’m thinking of doing for my fall course:

We’ll talk in detail about the writing processes of people who write a lot, myself included. I’ll probably share a sped-up screen recording of my own process (maybe the one used to write this post) and others, if I can find them. I may give students the option to record their own process if I can figure out how to make that work—but I don’t think I’ll require it.

I’ll continue asking students to draft their work in Google docs so that they and I have access to their version histories. But I will 1) be more transparent about the fact that I can see their version histories, 2) explain how they and I might use those tools as sources of reflection, 3) assure them that I’ll only dive into the version history if I’m seeing something in their essay that concerns me. It will always be a springboard for conversation about the writing process rather than a way to catch cheaters.

I’m also cooking up an assignment that asks students to look back at the version history of a previous essay and write a brief reflection on what it tells them about their writing process and how that process has developed in recent weeks. I’m hoping this shows them that I’m less concerned about tracking their work and more concerned about encouraging reflection on it.

I’m already thinking, though, about what will happen if students’ writing processes don’t lend themselves well to this scheme. What if they like to draft essays by writing them out by hand first or just by making extensive hand-written notes? What if they, like Leon Furze, prefer to dictate their essays, have them transcribed by AI, and then copy and paste them into a document for editing? What if they start by doing a lot of free-writing that is intended for their eyes only?

In all honesty: I’m ending this post less certain than I was before about how to balance the need for metacognition, the desire for accountability, and the concern about privacy in all of this.

What are your thoughts about process tracking? What are your plans for using it in your next class? Please share your own approaches in the comments.