Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well; many of them can no longer brainstorm, code, think deeply, or write without an LLM present doing 90% of the work. Many of them can no longer sit quietly for even 30 minutes just thinking on their own, which is a required skill for producing original thought.
For adults the cognitive decline won't be as measurable since there's no exams, and overall output volume will still be fine due to LLM help. But I do believe it's already happening absolutely everywhere around us. Honestly, I wanted to be in denial about it before but it's too obvious to ignore now.
However, I personally feel a huge mental burden of the state of communication. The contemporary version of it where I have a million threads and conversations im juggling at any given time. Emails, voicemail, chat, online, texts, personal, business, home, children, other family, friends, then there’s the variants like Messages, Messenger, WhatsApp, etc. And as overwhelming as it is for me, I’m super under connected than everyone else I know. I quit following most news and all sports, as I just don’t have the bandwidth for it.
My brain was molded preinternet and I feel like it’s reaching its max on the analog to digital conversion. Or at least it’s just a really lossy process.
I'm learning a new code base for a new job right now, and I'm finding AI to be a really double edged sword for it. One one hand, it's extremely valuable for asking questions about the code base. On the other hand, if I'm not careful and I just let it apply the fix before I even investigate it, I'm really not learning the code base well at all. I find I need to actually write new code in a code base to exercise the necessary mental muscles to actually retain understanding.
Incidentally, I do find that this large new code base I'm learning also shows the limitations of AI. There's no way I can vibe features on this without understanding and not introduce a lot of issues. Even targeted bug fixes have a lot of unintended consequences the LLM doesn't see. This isn't a bad code base at all, but it's definitely at the size where even frontier models struggle. So to me that tells me that the argument that I should just use more AI to solve my AI issues and not bother to understand the code base isn't viable at the moment.
I'm not noticing a "cognitive decline" per se, but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.
Something radical needs to be done. When I was in high school there were still a lot of "no calculator" restrictions in my math classes that I chaffed at because I hated doing longform arithmetic and felt like it got in the way of learning. So I can certainly understand how students would chafe at some kind of paper-only education system but I also don't see how you can learn anything when you have a high-quality homework machine just sitting there.
I agree - I would have been toast. I wonder if the teachers/colleges need to change the way they teach and assess. Let the students use the AI tools they like (perhaps guide them how they can use them professionally), but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person. Oh and don't give Fs for cheating - suspend them.
I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.
EDIT: Claude beat my Googling: This was 2 chemistry high school teachers in 2007 - The Flipped Classroom https://fltmag.com/the-flipped-classroom/
There’s no way to learn than to force the brain into adaptation which it is resistant to do through challenge and stress, just like your muscles. Similarly you can’t play e sports and get into physical condition any more than you can use LLMs to do your homework and learn.
It’s going to be a hard adjustment for a lot of people to recognize that letting the machine think for you is as healthy as smoking brain cigarettes.
The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button. They make great tools for learning if they’re used as an adversarial or editorial tool. The future belongs to those who work to use the tools in ways that make themselves more efficacious, not those who use efficacious tools so they don’t have to work.
If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.
The leading indicator for me is the amount of emails and, god forbid, more personal messages (like birthday wishes!) I see that are obviously AI generated. It just keeps on rising. If you’re not able to dash off a quick message without the help of AI I have to assume you’re using it heavily elsewhere too.
I have sympathy for the university students too, we’re all bombarded with rhetoric about AI being the future. And I remember being incredibly nervous emailing my lecturer (am I phrasing this right? Is it respectful enough?) that I can imagine leaning on AI myself had it been available back in the day. But I’m glad it wasn’t, it’s an important skill to work out this stuff. They’re going to land an in person interview when they graduate and stumble around unable to effectively answer the questions they’re asked in real time.
You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
So, as long as you are not under time pressure (which you in some degree courses unluckily are), there is simply no need to "speed up" any homework assignments.
If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study (which is only loosely correlated to homework and tests), I guess it's fine to use them. Just always keep in mind that very often the pain of attempting to understand the topic on your own often makes you smarter - something that you will miss when you take an "LLM shortcut".
Around COVID times many top universities experimented with removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere, with many, if not most, universities already reversing it. As Yale put it, "Yale’s research from before and after the pandemic has consistently demonstrated that, among all application components, test scores are the single greatest predictor of a student’s future Yale grades. This is true even after controlling for family income and other demographic variables, and it is true for subject-based exams such as AP and IB, in addition to the ACT and SAT." [1]
That link is for an archive because that page has been removed. That's because they briefly experimented with a new 'test flexible' strategy where they allowed students to submit test scores or not, but then scrapped that altogether and went back to simply requiring test scores.
[1] - https://archive.is/8zxfo
To fellow professors, when you're suspicious my suggestion is to appeal to their honesty (like "let's be honest, how much of this code is yours, and how much is ChatGPT's?") and offer some empathy and understanding (like understanding they may had multiple deadlines in the same week, etc.). Nevertheless, don't miss the chance to give them the lesson on how is the correct way of doing things. The way to catch these students is to find the same signs of yesteryear copying from other students (which in essence is what copying from an LLM is, although the number has increased because they found us professors unprepared for the volume).
The other two groups also used LLM but in a high-level and architectural way. They were clearly responsible for the code (even if they didn't wrote it 100% manually) and could explain their reasoning and strategies used to solve the problems.
Me and my colleagues still have a lot of projects to review, and I asked them to keep the score of the number of projects like these, but so far, the score is 1 in 3 (33%).
Do we assume society just self regulates. I think it does, but the cost of letting it self regulate is really really high, with lots of suffering. Is it that we find this acceptable when there is a chance we won't be the first to feel the pain?
"More than 600 University of California faculty members, led by mathematicians at UC Berkeley, are calling on the system to reinstate standardized testing requirements for science, technology, engineering and mathematics applicants, saying that six years of test-free admissions has not reliably assessed readiness and professors are often teaching middle school math to incoming students."
As a Cal alum, I am actually really glad to see they are holding the line on grade inflation. I worked my butt off to achieve the GPA I did, and it would really suck to see my labor devalued if Cal went the direction of e.g. Yale and started handing out 79% A's and A-minuses: https://yaledailynews.com/articles/professors-face-grading-d...
It worked, and it would have been MUCH harder to do this the traditional way.
The tool generates PDFs including an answer key and solution sets that solved the problems using a variety of techniques so I could check her work more easily and we could iterate quickly.
That's powerful. It comes back to how are you using the tool. Are you using it to make things better or to take shortcuts?
I was worried they may have cherrypicked courses that support their chosen narrative.
So I plotted the % of F grades (red line) for all CS courses still offered, and sorted the chart in descending order of the # grades given out (light blue vertical bars) in the most recent semester when the course was offered.
My worry was borne out. See the first few charts. No big increase in F % in the past few semesters.
It kinda was fun, like a very patient professor stand right besides you. It was the one of the best math learning experience I've ever had, and you don't even need to send bribe/gift to Gemini to keep you in it's favor.
On the other hand, if you ask a LLM to completely finish the work without thinking it through by yourself, then it sounded like cheating, to yourself.
My son is 15 and I use Google Family Link to control what he does on his phone: it's pretty open for the most part (I receive notifications of installs) but Gemini is a hard-ban.
We've spoken at length of the dangers.
He says his pals use LLMs frequently and I suspect that's the reason for their test scores: some of them are in the 20% - 40% range for tests whereas my son is 80%+ because he studies past-papers and answers questions in his revision.
I worry for the future coz you can be sure that the AI providers don't care if a schoolchild is using their LLM to answer the homework questions.
To do this, you have to be a professor who has a strong idea of what subject mastery looks like. Not available to most.
But ... It is exactly the right idea IMO
I think it's true that we collectively lose something akin to beauty every time technology advances. But usually some new set of skills that have beauty emerge.
If LLMs end up being the pneumatic nail gun for the human mind, I personally think that's a fine thing for us to accept.
If they end up being more like some dark factory that autonomously does everything - then I think ultimately the thing that makes us human (our minds) will slowly decay and be lost, and that seems very sad. That's a version of the future we should try to prevent, I think.
That was in the 1980s.
My first math exam as a CS undergraduate, 123 out of 129 students failed. The math department professors refused to dumb down their classes for CS students.
Math was core to the CS curicullum in those days. It would fade away over the next few decades to almost nothing. The main reason being the CS department wanted to popularize its uptake, and remove barriers that kept students from passing. There was also a major dose of interdepartemenral rivalry and academic politiking involved.
WAL-E and Idiocracy. The future.
Grade curves are how you test your curriculum for good challenge - are you challenging people such that an A isn't a too-low threshold. When you force people into a curve, you haven't defined a threshold of mastery, you've defined a sorting function: A means "better than this year's peers". It is absolutely bananas to me that a tech/math oriented school would be doing any sort of curving.
> You can always ask me for feedback on your homework and I will mark up every part of it, but you won't receive a grade for homework. However, if you don't do the homework and take your time with it, you will fail the class. My office hours are in the syllabus and you're strongly encouraged to use them. There will be an early exam to give you a chance to know whether you are likely to fail this class before you lose your chance to drop it.
Correctness is harder to adjudicate in some humanities disciplines but the format of these exams is actually not super different from essay tests (when a math professor grades a proof, they're inspecting specialized prose for validity, coherence, persuasion in a way that also reveals knowledge).
When you don't rely on homework for determining whether or not a student passes the class, you make cheating on the homework into the student's problem instead of the professor's or the university's. Students have the right incentives to solve problems for which they are the ones responsible, and they figure it out after one failed (or ideally, dropped) class at worst.
But I essentially completely stopped using them for software engineering (why isn't really relevant, but it's not because od this skill atrophy). So as the skills of everyone else is diminishing, mine is proportionally raising.
It has never been easier to get better than others. You don't need to put in more effort, just the same effort as you always have, and others will do the job of losing their skills for your own benefit.
As a naturally curious person, nothing will stop me from learning about the topics that interest me. But school also taught me a lot of things that didn't interest me, and a lot of those things turned out to be useful anyway. I think if I had access to AI from a younger age, I'd have used it to skip learning the things I didn't care about, which would not have done me any favours.
I'm feeling effects of using LLMs day in and day out, but I am not yet convinced that it's overwhelmingly negative, the way much of HN seems to lean.
I derive a lot of joy from shipping outstanding code for my clients and fixing problems they're experiencing. My joy has only increased as I can now ship better code, faster, with fewer bugs. No, I don't intimately understand the code the way I used to, but I understand it enough to accomplish the end goal.
The premise of this article takes a presumed position that the grades we were posting before really mattered a whole lot. I'm not convinced that's true.
That said, assessments of poor critical thinking skills jump out at me more than the rest. That sort of thing seems likely to matter until machines can replace us completely.
The future of education is rapid back and forth between the student and the teacher. You could image a future where a student sits down all day with an AI system that trains the student, and it won't let them pass until it's confident you know the material. This will happen at different rates for different people, but this is expected.
An ideal education would be getting those skills discs uploaded to you in the Matrix, but unfortunately it will take a bit longer to master them as we don't have full brain-system interfaces (yet).
It will have taken us less than 1000 years to go from scarcity of the printed word to the over-abundance, and finally to the uselessness of it.
I was in my 3rd bachelor's year studying physics (France) and overheard a conversation between two of my teachers. They were discussing how they should modify the 1st year program to now include math, because he had been noticing how more and more students were failing the more math-heavy subjects like body and newtonian mechanics. He said that they should now teach (or re-teach) calculus to 1st year students, which was not taught when I entered college (it was assumed that you learned it in high school and we would only cover linear algebra in 1st year).
I can imagine things are only getting worse with students that can now get under the illusion that they know math because they have a tool that can do it for them. Which raises the question: should programs adapt to this, like we adapted to having calculators?
It’s that there is no reward for doing so and in fact there is punishment.
The punishment is that for all the thinking you do, someone else will arrive at the same result as you in less time, or maybe even a better result. You don’t get rewarded for the effort of thinking, only for the end result.
Naturally, even if you are an intelligent individual, you can still be conditioned in this way to take the easy way out, unless you purposely like to suffer. But suffering is only worth it if you know in the end you come out ahead.
But now, you do not come out ahead. People will be using AI in the workforce for the rest of your life anyway, might as well just join the trend.
It’s like if everyone started taking a magical steroid and growth hormone to build muscle and look great instead of actually working out in a gym and possibly getting worse results anyway.
Artificial Intelligence and Grade Inflation
https://cshe.berkeley.edu/publications/artificial-intelligen...
I TA’d in the early 2000s and the first day students were warned that we used automatic analysis to find programming assignments that were similar to previous submissions. And renaming things, moving them around etc would not help.
We caught and failed cheaters every term.
So learning was never the actual goal.
Originally, at least in premis, it was to learn and advance the arts and sciences.
So what we need now is a college for llm's to advance the arts and sciences.
The solution? I'm not sure but possibly use AI as more of a collaborate partner to discuss with rather than letting it give you the answers
So the Claude web app has this “learn” option that turns the session into a Socratic dialog of sorts. One could easily imagine enforcing this on an age based or parental controls set up. Maybe it can be prompted around but at the very least the concept could be a path forward.
As others have said there is a way to use llms to increase learning, but autodidacts will always autodidact.
A bunch of science fiction stories had "first connection to cyberspace" as a coming of age event, maybe those authors were on to something.
The goal of education is to impart knowledge in the student, preferably correct knowledge. The goal of an LLM is to produce an output that is convincingly human. It's not even that they're opposed, as much as they're ships for whom Polaris is in a completely different direction.
"Hallucinations" as they're called, or more plainly stated when the machine makes some shit up, are perfectly understandable in this context, as are the struggles of every single AI firm to get rid of them. Namely: the machine is functioning exactly as it is designed to, so how can you possibly fix it? It's working. The goal of an LLM is to produce text that passes for human, and apart from the obvious LLM tells, it largely does. Like say what you will about their lack of intelligence, the writing is solid. It's grammatically correct, spelling is dead on, what have you.
It reminds me of the famous phrase from Chomsky: Colorless green ideas sleep furiously. A sentence which is perfectly grammatically valid but is also completely devoid of meaning. An LLM would write that sentence, and it would be working correctly.
All of that to say: for all the things they CAN do and CAN be used for, I think we have to draw a hard line at education. I just don't think AI has a place in it. Of course that presumes that the goal of education is to, well, educate people, and especially here in the States but also abroad, we have been putting other interests, especially capital, far ahead of that for decades. I expect no different here.
And before someone comes in to go "WELL HOW DO YOU THINK YOU'RE GONNA STOP IT LUDDITE IT'S THE FUTUUUUUURE" yes, I'm sure as long as these exist and are available to people tech literate enough to access and use them, whatever that means into the far flung future, they will be a factor. Just like cheating, just like plagiarism, just like everything else that will get you kicked out of school. And the answer is the same: it will be stopped by institutions, imperfectly, and it will also happen anyway and with the same consequence: those responsible will mostly be harming themselves for short-term gains.
2. No US educational institution should ever grade on a curve. Your job is not to compare students but to educate them. Grade curves hide the performance of the educators and process of education in actually improving the skills of students.
3. Both AI and the cognitive and emotional overload from social media taking away brain space may be to blame. Idea: let students report screen time statistics at the beginning of each semester and weekly or at the end. See if and how it correlates with academics.
You'd have to be really lazy and disrespectful, to get caught cheating on a take-home exam...