How I changed teaching after AI managed to do all my homework assignments

I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.

I rely on written exams that are open book, but forbid any use of computers and smartphones in class.

The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.

I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.

It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.

Seems the core problem is that many/most students aren't there to learn the material, just to get grades. The class is a complex mechanism to force them to learn material against their will in a validation arms race. The teacher's job becomes something like a military commander. Seems depressing.

I have a fairly long experience of both teaching and grading, even though I stopped a couple of years ago. After a couple of years of trial-and-error, I think I found the "right" way:

* Alternate between theory and practice in the course of a single 2-hours slot. Theory is necessary, but the attention span of people is very limited. * Grade either a homework assignment in the form of a real project with specifications. You give the assignment half-semester, so that students who start early can ask questions. Otherwise, grade a in-session exam, with every resource available, including the course and the whole Internet. However, you don't assess the knowledge, but if the student is able to apply their knowledge to the different tasks at hand.

These approaches are now completely moot in the age of AI.

I was surprised the first time a colleague asked me to do an oral exam. I thought it was a burden on people with poor social interaction skills. It went surprisingly well: you could see in a couple of minutes if the student had integrated the concepts of the course, even if they were shy. Now I wonder if it's the way to go. There's one caveat, though: it doesn't scale.

> I now shifted to in-person interactions with a TA. After every assignment, each student needs to schedule a 15-minute meeting with a TA to answer a couple of questions in a live conversation

Isn't the answer the 'flipped classroom' - students are assigned reading/studying to do before a class and the class becomes more interactive, answering questions/discussing topics/solving problems etc depending on subject. Of course, this is more expensive than a hall with 400 people listening to a lecture with a couple of tests/essays.

https://fltmag.com/the-flipped-classroom/

https://en.wikipedia.org/wiki/Flipped_classroom

It's funny how people talk about "teaching" getting more difficult with LLMs.

Clearly teaching got a lot better, since we now have a new teaching tool at our use, the LLMs. It's evaluating that got harder, because students can use the same tool to cheat in traditional forms of evaluation.

If "teaching the students" means the same as "evaluating the students" to you, maybe you shouldn't teach in the first place.

Thanks for sharing. Interesting to see how teachers are adapting to LLMs. I agree that trying to ban AI use is futile.

Maybe the Germans were right? One in-person test, preferably an interview, at the end of the semester should be enough to motivate the smarter part of the bunch to make sure they understand the issues at hand. The rest is chaff anyway. Don't optimize for chaff.

Thinking too small. The interactive testing needs to be done by AI. Then the transcript is reviewed by the teacher.

We still have developers because we can't outsource responsibility to AI. If shit breaks, we still need someone to fix it. Maybe you think that can also be automated. I have a counter-example. AI will rarely use the existing state when solving a bug. It will tack on it's own event queue, it will make up it's own messaging system, it is a cancerous growth. And this observation is based on both opus5.5 and gpt5.6-astra. The results are way past magical at this point, but you still have to weigh in pretty heavily for orderly code growth where solutions are succint, focused and preserve the existing structure. They are still noobs at fixing/adding stuff.

This makes me think of exams — I always did well on open-book exams, but I didn't do well on exams where I couldn't look at the book. Whether it's because of ADHD or just a lack of memorization ability.

Similarly, I did well in interviews where I could use the internet, but I didn't do well in interviews where I couldn't use the internet.

Everyone has a testing format they're good at. I'm curious about how this kind of thing is evaluated, and whether there are any papers related to this.

Just grade the process, not the result. It's literally that simple.

AI can talk the talk, but it can't walk the walk.

A simple "explain why this is the best solution" will make it shit the bed 10 out 10 times.