back

by helsinkiandrew·2mo ago·view on hn ↗
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.

I agree - I would have been toast. I wonder if the teachers/colleges need to change the way they teach and assess. Let the students use the AI tools they like (perhaps guide them how they can use them professionally), but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person. Oh and don't give Fs for cheating - suspend them.

I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.

EDIT: Claude beat my Googling: This was 2 chemistry high school teachers in 2007 - The Flipped Classroom https://fltmag.com/the-flipped-classroom/

4 comments
Absolutely university has to change. But it's not a simple change. I say this as a professor for Physics:

My colleagues say "We must fully embrace AI as a tool". I agree. But how do you teach it? It's a moving target, and you can't even give homework like: "Research <this topic> with an LLM of your choice, and submit the transcript" because they can do that, or they can just copy the task into an LLM and have the LLM do it. It becomes meta quite quickly.

And independent what and how we teach, we have to change how we assess a students learning result:

The first thing we have to change is that homework needs to be completely ungraded. Reviewed and corrected, yes, but not part of the grade. That's the only way to make sure that people who don't want to cheat have to cheat anyway to compete with those that do.

Second, all exams have to be in person. Online, cheating is so trivial it's not even funny (many students are so stupid about it that we have a pretty clear idea what's going on). In person, we have maybe 2-3 years until we have to make sure its proctored and people's glasses are checked. I think in less than 10 years, local mobile AI will be good enough so even a Faraday cage will not help.

Maybe we have to go to oral tests only.

Of course, none of this scales. Some of our intro courses have a thousand students.

Any ideas are much appreciated.

I feel like German university was already well "LLM proof". Come to class or not, no ones keeping track and it doesn't matter for grade. There are regular exercises, but they are not graded; you can submit them if you want it corrected to check your understanding, but there is also gonna be a short presentation going through it in class. Your entire grade is a 2-3h written exam at the end, no materials, no remote, no books, no multiple choice.
That's how it used to be. At least at my alma mater, it changed a lot when they switched from Diploma to Master/Bachelor
> Maybe we have to go to oral tests only.

"Perfect homework, blank stares: Why colleges are turning to oral exams to combat AI" https://apnews.com/article/college-oral-exam-ai-chatgpt-7795...

>Of course, none of this scales. Some of our intro courses have a thousand students.

Any ideas are much appreciated.

Oral exams graded by LLMs? Scale with the improving models. Based on GPQA Diamond results they're mostly at PhD level for subject trivia anyway.

The problem here is that it will work for now, but how do you make sure the LLM talks to the student, and not a different LLM? I guess vision models FTW?

In the end, will be build a GAN loop?

Why am I now reminded of corewars?

>(perhaps guide them how they can use them professionally)

If that's anything like how they guided me to use programming languages professionally...

In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial. They've barely adapted to video conferencing.

How do they keep up? Honestly it’s too risky to make a serious move on such a fast-paced environment.

The only thing I can see them doing is removing technology altogether. People did just fine 100 years ago.

Want to learn to code? Use a Commodore 64. The company was purchased and rebooted the C64: https://commodore.net/

Yes, meanwhile, Claude Cowork was only released this past January. And that was amazing. But I don't know about anyone else but I've already moved on to just using Codex for just about everything (except some Kagi use). Schools work on timescales of years, AI is advancing on the timescale of weeks and months.

Until that situation stabilizes I think the only institution capable of teaching about it is the family -- parents.

I'm not sure parents have the right tools either. Microsoft is about to ship OpenClaw as part of windows (talked about at BUILD) and they're acting like it's production ready and they've solved the security issues.

I don't believe them for one second, it's far from a solved problem yet these companies are selling this tech as if it's been around for decades and thoroughly battle tested instead of highly experimental and unstable.

> In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial.

Perhaps this is rather a sign that you currently shouldn't jump on the LLM hype train, but rather attempt to get a good foundation on the basics. When the whole LLM area becomes much more "stabilized" (I see signs that this is currently happening, if only for the reason that training state of the art models has become more and more expensive), you can still get into LLMs if you want.

I partly agree, depending on what you count as the basics. I don't think there's much value in learning the quirks of LLMs today: they will just change, your value-add becomes part of the model or harness.

On the other hand I think there are real development gains in jumping on the train today. To my career's detriment.

Traditionally, moving slow with policies was fine with new tech because, outside of the PC revolution it wasn't all that impactful, and things used to rightly be labeled as experimental so you could safely ignore it for a while as a big enterprise and be just fine until thinks shook out.

LLMs were, IMO, pushed out too early and without that clear "this is experimental tech" label. Full public access from day 1, no invite only betas, no research previews for a select few pilot customers/orgs, etc. I've been in IT for a little over 18 years now and I haven't seen anything move this fast before.

I mean, I never though I'd see Microsoft go on stage at BUILD and and announce freaking OpenClaw for Enterprise, and then make it available the same day. This is highly unstable tech and what I'd consider still experimental, being sold to F500s as production ready.

Ancient brains, medieval institutions, godlike technology.

Tristan Harris had some sort of comment like that on a podcast about the challenges posed by AI.

I think it's incredible how much those ancient brains can successfully adapt to technology. Some people can sit in highly-strung sports cars and use them at the absolute limit of their performance like they're just an extension of our own limbs and senses.
> I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.

That seems like a smart approach. It reverses the traditional model of "lecture in class, homework outside of class".

A smart approach that does not solve the AI problem - actually flipped classrooms work worse now due to AI usage.

My own experience with flipped classrooms (which seems to be shared by quite a few people who have tried it out): they only work well if all students actually read/watch the materials beforehand. In small, advanced courses, intrinsic motivation may be sufficient - but in most cases you need some extrinsic coercion - such as a mandatory quiz about the materials or hand-written lecture notes that need to be shown at each in-person session.

With AI, some people don't watch the lectures but let ChatGPT give them a summary which they submit. Then these people poison your in-person session with their lack of knowledge and motivation.

Research has shown that testing is far and away the most valuable academic tool.

Just have a quiz every day. In fact, have _TWO_ quizzes, one at the start of class and one at the end, and take the higher of the two scores. In between the first quiz and the second, work through problems with the students designed to help people that bombed the first test figure out how to pass the second.

The best part of a quiz everyday is that in addition to the testing effect, you can easily fit in the spacing effect and interleaving effect. It’s a rock solid combo, that is well studied. We have pretty strong evidence that it works for all students in all domains.
I actually like this idea - makes sense at face value - as long as they design the test in such a way that it aptly applies the knowledge instead of just learning for the sake of passing test like questions...
I had a flipped classroom for my topology lecture. It was one of my absolute favorites.

We had no lectures, the teacher just gave us a short, concise textbook to read a chapter of every week.

In class time was devoted to discussing and problem solving.

But yes, it only worked because we were a small class of 15 math students

My Cryptography professor did this during COVID, since the classes were split in person. It was an interesting model. I'm not sure if I loved it or not, but it was at least a change of pace. Getting 100% of the class time to ask questions was really nice, but it ended up with him re-teaching most of the online lecture in class because some quarter to half the class just didn't watch the lectures.

If done more stringently (if you didn't watch the lecture, I'm not reteaching it), it maybe would've had a bigger impact, but I'm not sure.

Office hours remained king for serious Q and A for the class.

One way to fix that issue that I’ve seen is a daily quiz to start the class. The key is the quiz is super easy. Even if you were confused by the lecture, if you watched it at all you’d likely get a 100 on the quiz. If you didn’t watch it you’d likely get a 0. This quickly for people watching the lectures online ahead of class.
Flipped classroom pedagogy has been the subject of a huge amount of research. Ultimately "one weird trick" solutions don't tend to work in education. Enough students don't watch the lectures that you end up needing to go over the material in class anyway. Funding and autonomy works, but nobody likes to pay more.
I always absolutely hated when a teacher did a reverse classroom and I had to “learn” at home and then practice in the classroom. I think the solution is more engaging lessons and less outside work. I know why homework exists, but homework is a chore that most people want to get done as fast as possible. If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI. If they’re interested in the topic, they’ll put more effort into it. If not, they were never going to retain it anyway
When left to their own devices, 99% of children are not interested enough in math, history, literature, languages, or almost any other school subject to engage with it willingly. Only teaching "interesting" things that kids are "interested" in is both impossible (too many varied kids per class for that to work 100% of the time) and even if possible, would leave kids with zero practical knowledge, because learning most of that stuff is not something kids inherently want to do.

College is different, because theoretically you should be taking classes that are relevant to your field (although there are still "core" requirements that are somewhat high-school adjacent).

I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.

College is a different dynamic from a middle/high school classroom, but I don’t remember 95% of the material from my college engineering classes anyway, it’s the problem solving and information finding that I’ve retained and have helped me do the things I do. I remember the stuff from the classes that taught me the material in an engaging way though.

>I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.

"Just do it right and it won't be a problem." This is not an actionable plan. What is engaging? Who gets to decide that? The teacher? The students? The parents? How do deal with certain kids finding different approaches more or less engaging? How do you expect a teacher to curtail their teaching approach to dozens of children at the same time?

> homework is a chore that most people want to get done as fast as possible

Worksheets certainly are. But good homework, even if it's challenging, is what makes a reasonably fast-paced course even possible. In a well-paced university course you're typically spending proportionally several times as much time working on it out of class than you are in class. Then class time is both preparation and catch-up, similar to office hours.

This was true of my most demanding humanities courses (sometimes reading 100 pages a week directly from academic journals, not easy reading) as well as my most challenging math courses (group theory, ring theory). Once the pace gets fast, there just isn't enough time for you to learn everything you need to inside the classroom anyway.

And in those classes, where homework was really essential for learning at the required pace and depth of mastery, my instructors didn't even need to factor the homework into my grades at all. In some of them, we could get "feedback" on homework but it was never officially recorded in our grades... and yet, anyone who didn't do it would fail the next test. If homework doesn't have that characteristic, it probably doesn't need to be assigned at all.

If "flipped classroom" means that students are expected to do all of their homework in class, then indeed it'll feel like a waste of time to many of the smarter kids, and it will also just be unfeasible for advanced courses (which theoretically should be most courses in a university, though it currently isn't). But if it means "we don't even have time to lecture you on every single thing you need to learn, therefore you must arrive already having done the reading and the exercises, and we'll use this time to help clear up misunderstandings"... that's already how classes for grown-ups are in universities.

>If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI.

Kids get to learn lots of interesting things in school. The problem is that they're kids! They want immediate gratification from phones/games/recess, not to do the hard work of learning.

How does this scale in practice? We already require students be at school for 7 hours a day. If they now have to watch 3-4 hours of lectures at home every day, then students are left with little time to do anything else.

What about those students who don't have stable home environments? How are they supposed to find multiple hours a day to watch lectures?

How does this address the underlying issue of students off loading work? You've replaced homework with lectures, but haven't solved the problem of making sure the student is actually participating.

Logistically, this could only work if you shortened the school days, but then you would need to adjust the rest of society around that. Many parents structure their work days around their kids school schedules, and if kids need to go school later in the day, or get out earlier, that places a burden on the parents.

We’re discussing university, right? It’s supposed to be a full-time effort, at least for a normal pace undergrad or any post- graduate program.

For secondary school, I do agree with you - homework load can be problematic for some students. But at the same time, my honors classes all came with hours of homework and I’m not sure I would have been as prepared for uni without it.

From my experience it works fine if it's one class that's doing it. If multiple classes are doing it then like you said it's literally a couple extra hours a day watching lectures and most students end up skipping them, forcing the instructor to end up teaching during class time anyways.
> Let the students use the AI tools they like […], but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person.

I very much doubt there is any agreement on what those skills are.

Creating the idea of “what to learn in the new world” is itself IMO an important academic creation, but there’s no reward for doing it and no way to know if you’re on the right track (you just have to wait and see).

Employers are also just adapting.

Wait until companies are paying unsubsidized “list price” for LLM usage. Then we can have a better idea of the worth of the automation and what skills should stay with humans.

This. The industry is dumping electrical labor at a humongous loss JUST BECAUSE they figure people will immediately atrophy and be unable to do without AI… at any price.

We'll get an idea of the relative cost of the labor, all right. It's just that they are specifically trying to wreck the market, at all costs, to be able to cash in on the upside. It's sensible, if you're a monster.