Failing grades soar with AI usage, dwindling math skills in Berkeley CS classes

However, I personally feel a huge mental burden of the state of communication. The contemporary version of it where I have a million threads and conversations im juggling at any given time. Emails, voicemail, chat, online, texts, personal, business, home, children, other family, friends, then there’s the variants like Messages, Messenger, WhatsApp, etc. And as overwhelming as it is for me, I’m super under connected than everyone else I know. I quit following most news and all sports, as I just don’t have the bandwidth for it.

My brain was molded preinternet and I feel like it’s reaching its max on the analog to digital conversion. Or at least it’s just a really lossy process.

I'm learning a new code base for a new job right now, and I'm finding AI to be a really double edged sword for it. One one hand, it's extremely valuable for asking questions about the code base. On the other hand, if I'm not careful and I just let it apply the fix before I even investigate it, I'm really not learning the code base well at all. I find I need to actually write new code in a code base to exercise the necessary mental muscles to actually retain understanding.

Incidentally, I do find that this large new code base I'm learning also shows the limitations of AI. There's no way I can vibe features on this without understanding and not introduce a lot of issues. Even targeted bug fixes have a lot of unintended consequences the LLM doesn't see. This isn't a bad code base at all, but it's definitely at the size where even frontier models struggle. So to me that tells me that the argument that I should just use more AI to solve my AI issues and not bother to understand the code base isn't viable at the moment.

I'm not noticing a "cognitive decline" per se, but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.

Something radical needs to be done. When I was in high school there were still a lot of "no calculator" restrictions in my math classes that I chaffed at because I hated doing longform arithmetic and felt like it got in the way of learning. So I can certainly understand how students would chafe at some kind of paper-only education system but I also don't see how you can learn anything when you have a high-quality homework machine just sitting there.

I agree - I would have been toast. I wonder if the teachers/colleges need to change the way they teach and assess. Let the students use the AI tools they like (perhaps guide them how they can use them professionally), but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person. Oh and don't give Fs for cheating - suspend them.

I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.

EDIT: Claude beat my Googling: This was 2 chemistry high school teachers in 2007 - The Flipped Classroom https://fltmag.com/the-flipped-classroom/

If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.

There’s no way to learn than to force the brain into adaptation which it is resistant to do through challenge and stress, just like your muscles. Similarly you can’t play e sports and get into physical condition any more than you can use LLMs to do your homework and learn.

It’s going to be a hard adjustment for a lot of people to recognize that letting the machine think for you is as healthy as smoking brain cigarettes.

The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button. They make great tools for learning if they’re used as an adversarial or editorial tool. The future belongs to those who work to use the tools in ways that make themselves more efficacious, not those who use efficacious tools so they don’t have to work.

The leading indicator for me is the amount of emails and, god forbid, more personal messages (like birthday wishes!) I see that are obviously AI generated. It just keeps on rising. If you’re not able to dash off a quick message without the help of AI I have to assume you’re using it heavily elsewhere too.

I have sympathy for the university students too, we’re all bombarded with rhetoric about AI being the future. And I remember being incredibly nervous emailing my lecturer (am I phrasing this right? Is it respectful enough?) that I can imagine leaning on AI myself had it been available back in the day. But I’m glad it wasn’t, it’s an important skill to work out this stuff. They’re going to land an in person interview when they graduate and stumble around unable to effectively answer the questions they’re asked in real time.

You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.

So, as long as you are not under time pressure (which you in some degree courses unluckily are), there is simply no need to "speed up" any homework assignments.

If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study (which is only loosely correlated to homework and tests), I guess it's fine to use them. Just always keep in mind that very often the pain of attempting to understand the topic on your own often makes you smarter - something that you will miss when you take an "LLM shortcut".

They stopped requiring SAT and ACTs in order to get a student population more representative of the population in general. This obviously allowed students that were not prepared for college into the system.

If you do well in your math SATs you'll likely do well in math college. SAT scores and college GPA are highly correlated. No idea why anyone thought it was good to ignore probably the strongest signal of success in college.

But this doesn't seem to make sense when someone comes to a topic with an LLM in-hand. They need to know high-level techniques, architecture, best practice, etc. As they pursue the topic they start to get down into the details, although probably never learn to do it fully independently.

I quite like this view because it paints a somewhat optimistic way forward from where we are now.

With writing:

Things like brainstorming a plot line for a book with a custom GPT or Claude project that has all of my prior books in its knowledge? Works great.

Things like asking it to write a paragraph or chapter for me - I can rapidly feel my own writing skill, motivation, vocabulary, and ability to grasp/remember the resulting plotlines deteriorating. I don't use it for that anymore.

With studying:

I've been taking a couple of evening uni courses and the thing I found so great is that I've been forcing myself to think through the problems, and take my own notes in every lecture. I may then still get ChatGPT to help explain and reason through some of the concepts with me. And I have it review and 'grade' my assignments. But I refuse to ask it to start drafting answers.

With programming:

This one is tougher. When I am not very personally invested in a problem or codebase it becomes too easy to offload more parts to Claude, and when the company encourages 'vibing' to speed up velocity and you're reviewing and writing a higher influx of lower quality PRs, investment goes down. I still sometimes catch myself committing solutions I only _mostly_ grasp and the rest is hand-waving. A big part of it is a work culture thing.

For my own projects I make sure to understand and have a back-and-forth with the planning agent for each task, or write the first plan myself to go off of. When it comes to producing the code, I have to admit it is much easier to properly review parts of the codebase I am extra interested and knowledgeable in (backend in my case). The frontend I'm less well versed in and also admittedly less interested in, so I do sometimes fall into the trap of "Ehh it works, just commit it" with the goal of doing a thorough quality pass before actual release.

With all of the above, I can feel my ability to think, plan, reason, focus (and my vocabulary) suffer if I go over the line too much into agent offloading. For me keeping that balance is as much about maintaining my own long-term brain health as it is about producing good output. I imagine younger people growing up with AI today won't even know what that more capable (in my opinion) brain state feels like - to them, the AI-using brain will be the norm.

As a piano player, it’s important to work hands separately. Sometimes your right hand will carry the melody and your left hand the harmony, sometimes vice versa. Sometimes there may be more than just two “voices”/melodies/lines between your two hands. Even as a very good (as in getting paid to do it) sight reader, I learn a lot working all the voices/melodic lines separately.

Singers do similar things like singing only the vowels to keep themselves in the right placement. Learning handstands, you have to work your wrists, rotator cuffs, core (which is many things), etc. separately. Yoga, Pilates, and running also help us learn to break problems down this way.

Anyway, all that to say: If LLMs are gonna be a natural extension of how we think, we need to understand what parts of problem-solving LLMs are good for, and what parts our brains are for. The nice thing about working these bits “separately” is that one side is done for us. So we just need to consciously practice using our brains.

As programmers that means, maybe we conscientiously practice writing things ourselves sometimes. Remembering that this even if this sacrifices short-term “velocity” (whose measurement is problematic, but I digress), it preserves our long-term ability to do good work. And I think any of the above physical/artistic practices (or countless others), worked in these ways, will help reinforce this entire mindset.

I think kids of the coming generation will be sharply divided on their ability to conscientiously practice things separately. It’s been happening, but I suspect LLMs will accelerate it unless how we actually teach kids can catch up.

If we allow lying, cheating and stealing - why bother being a schmuck that does the work.

I believe this is the real crux of the issue. We often turn the target to things like "Can johnny Add, Read a book, or recite dates" which are only proxy measures for important things like "Can johnny solve a numerical problem presented to him, can he synthesize information, or can he think critically about what is occurring around him?" .

If students use AI to accomplish goals I do not see it an issue. If they cannot figure out how to use tools, or what their goals are-- that is a major issue!

An analogy of my point is that I don't want to focus on cursive in the age of computers keyboards, and I dont want to focus on abacus skills when a pocket calculator is like $5.

Plummeting attention spans has been a trend for much, much longer than LLMs and is more the result of constant digital interruptions and these days overwhelmingly social media and doomscrolling: https://www.apa.org/news/podcasts/speaking-of-psychology/att...

The effects on children have gotten most of the, err, attention, but the effects on adults are no less deleterious.

These are just a few examples of how I use LLMs to learn faster.

LLMs are a tool and I have seen massive gains from them personally

I think this is true of every affliction that adults criticize children and teenagers of

I’ve been out of university for a very long time, and I took a community college course and for the first few sessions I couldn't focus or sit still at all. Fortunately I knew that was abnormal and how to conform to a prior version of myself, but I don’t think children have a frame of reference.

A lot of skill of is getting bled into the private sector because getting the PhD in a lot of regions doesn't mean the step up it used to. A lot of that comes from awarding them to layabouts doing "a gender critical analysis of ...".

Industry doesn't how/what/why they just wanted the 3 letters as a performance barrier to hire competants.

That said, though, one thing I don't understand about the heavy users of AI in academia and software development is that the thinking and coding is the fun part. And that's the part so many people seem to be so keen to automate away.

Maybe the problem is that doing assignments contributes to your grades? The answer from wolfram alpha wasn't so much to get the homework done, but to understand how I would be screwed in the exam.

Now, if you’re creating trivial, unstable, or nonextendable systems maybe this doesn’t apply. And maybe I have long overestimated the work that SWEs have done.

As in I wrote code to generate random exercises, with solutions, using many tricks, to get myself hundreds of problems instead of 1 or 2.

Often spent more time on getting these programs right than on the problems. Still did better than the class. Oh and it was AI in the 1980s IBM sense. Ie. it was based around a python version (which I wrote) of a LISP math system based on maple. I even attempted (and largely failed) to rewrite it in C++.

Even attempted to have my homework read to have the computer correct the actual pages, but I never got convnets to reliably read entire lines (yes, I understand, well now, why a convolution would mostly not realize whether 2 pieces of text are on the same line or not and so get very confused if you go deep enough for recognition to work well)

As a counterpoint, I was once a physics grad student. I didn't finish the PhD because at some point I discovered that I was not going to be the next Richard Feynman and this was too much for my ego at the time. But I think that if LLMs were available, I might have finished.

Part of my problem was that at some point the math transitioned from stuff I understood to symbols and notation that I knew how to manipulate but didn't really understand. LLMs could have helped bridge that gap.

On the other hand, it's hard to imagine I wouldn't have used it for Jackson, etc. but we got Jackson solutions from previous students and the internet anyway. Using LLMs probably would have been more effective, used correctly.

The problem is that it sounds like many people are just using it for everything.

I noticed this before LLMs became a thing. It was by accident. We had a team of programmers. All decent at what they do. The management said 'hey you want to learn another language we are going to be using it for these upcoming projects'. So we set up a self learned at your own pace class curriculum. Maybe 10-20 hours of school work if you sat and really dug in. Maybe 3 to 4 hours if you breeze thru it and do not care much. We set up weekly check-ins doing about 1 hour a week. Easy. Watch a 20-30 min of vid 20-30 mins of do homework come to check-in and talk about what you learned and help others if needed.

Now this is where I was disappointed. The first 'class' was 40 people. By the last there were 3. Those 3 I noticed always are the ones who dug in. The rest wanted a proctored classroom and someone to tell them what to do.

Actual genuine curiosity is rare I think. We have a lot of people who are decent at what they do. But do not really care about it. IF you do not care you are going to just push the button and get the answer.

Asking suggesting or arguing to go deeper is impossible. There is a new path of least resistance and it saddens me.

The Whispering Earring: https://croissanthology.com/earring

I can still read code and write it, I just need to look back at docs a lot more, when I used to just know things. I also have to sit and try to recall how to do things and what abstractions are involved more. I also have more "writer's block" when starting with a fresh program/document if trying not to get AI to seed it with a baseline implementation, where I have to sit for a while thinking about what I really want to build.

At least now we know why we will start watering our plants with Brawndo.

I still did well, but I had gaps for which there was no help outside of the internet available.

(* I can count on one hand the number of time I've used an AI tool.)

Use-it-or-lose-it is the evolutionary principle, both for cognitive and physical abilities.

where do you see kids? This is a university. These are adults. 100% their fault.

tomorrow most regular people's thinking skills will definitely be weaker than those of the LLMs of tomorrow. And physical skills in most cases will be weaker than those of the robots. That leads to the question - what would most people do?

This was my experience even pre-LLMs though (about my own PhD thinking skills too). I blame the amount of random stuff work now involves more than LLMs.

I graduated from RPI with a degree in Management and a concentration in Information Systems. I began in Computer Science, and didn't like it because RPI CS at the time was loaded with professors who were mathematicians who had transitioned over to CompSci and because the 100 and 200 level courses were excessively math-heavy in my view.

Since this was the late 80s, there may not have been an easy way to teach B.S.-level computing without it being heavily math-based, but I digress.

No matter what degree we achieved or what work we ended up succeeding at, we have a tendency to look back at people rising in the ranks below us, see differences in their experiences and struggles, and say, Look! That is evidence of a lack of rigor or a lack of understanding of fundamentals that we had to learn in order to succeed.

The only thing is that some of what we learned to become successful just isn't necessary to be learned when we learned it.

I do a fair amount of low-level software engineering with Claude Code now that was above my level of understanding of data structures and algorithms because I never took those CS courses at RPI because I switched to Management IT.

But as someone who could be described as a solopreneur at some level, my new system designs reach a certain level of complexity or code maturity, and I hit problems that I would not hit if I had more understanding of data structures and algorithms.

So-- I end up having to learn aspects of those disciplines at that point, rather than before I actually needed them.

I run into these situations often enough where I now say to myself, gee, I wish I had taken Data Structures. And I think, could I effectively take Data Structures at this late date and get better at specifying how I want data stored, or perhaps knowing the shortcomings of simplistic database structures that are the ones I end up with initially because of my lack of spec-writing skill?

Aren't many of the less experienced folks who come up now, whatever age they are, going to hit problems that show them their weaknesses in this fashion?

Is the issue that these people will never get jobs because the seniors and managers who are interviewing them will design interview questions that keep people with their level of understanding out of the workforce?

What happens when somebody who sucks at the fundamentals but is really motivated bangs their head against their shortcomings and eventually succeeds in building something that takes off? Aren't those people great assets because they learned some of their critical skills the hard way?

1 - When I was in grad school (before AI), we had to use Canvas for a class. One day, I got an obvious spam/phishing email in the internal Canvas system. It was so strange. The writer just would randomly hit the capslock button and keep typing away, no salutation, no signature, just a real mess. They were asking for a particular professor to come to their house to teach them about ... something? Again, real strange.

So, I email IT and say 'Hey, somehow a spammer got into the system, do your thing'.

They email back and go 'Nope, it's a student, that somehow managed to CC the entire system, sorry about that'.

Dear Reader, the message was pure garbage. Literally, it looked liked it was written by a 3rd grader without any shame. [0]

I happened to know the professor of the class. So later on, I talked with them over symposium coffee about it. They said that they remembered that particular email because of all the IT back and forth. It was for an upperdivision class in the Engineering department. The email itself was not particularly notable otherwise. In that, they saw such emails all the time, in terms of quality. This was a top 100 ranked (whatever that means) university, by the by.

Shocking.

2 - My grandfather was an officer and a mechanic for the USAF. A bit of an odd combo, but he was partly responsible for instituting many preventative maintenance checks and protocols, novel in those early days of the AF. His aptitude and memory were quite sharp for many mechanical things. Until the strokes from decades of smoking caught up, he could tell you exact measurements and torque values for a variety of airplane related things (I can no longer remember what exactly, the memory skills did not transfer to me).

I do vividly remember standing in that light blue garage of his and him all but yelling at me once. We were looking at the brakes on an old car he was 'restoring' (getting away from Grandma for a little bit). He pointed at the old drum brakes on the axel.

He asked me how tight the pads should be on the inner rim of it.

I had no idea.

So he asked where I might find out.

I figured I'd ask him.

But what if Grandpa wasn't there?

We'll I'd have to look it up somewhere (they had no internet).

Fantastic. Now, what about the next time you're working on the brakes?

Well, just make sure that the pads are at that spec.

And that when Grandpa hit me with the nugget of hard won wisdom: No, you look it up every time. Because these are brakes, and if you are wrong then they might fail, and they might fail when the driver has their whole family in the car at 100 mph. And then because you were lazy, half a dozen people die.

---

These two times stand in my head when it comes to AI.

For the first one, yes, AI would be such a boon to that very clearly struggling student reaching out for help. It would get them back on the path to the real struggle of getting their degree. That level of assistance would be like a wheelchair to a paraplegic.

For the second anecdote, AI is condemning people to death. Using it in life critical situations and care, letting it hallucinate or skip over critical values, that's a recipe for disaster.

Where do we set the fine line of using AI and not? For brakes and X-ray machines, obviously not. For helping kids learn to write emails correctly? Sure, sounds great.

Unfortunately, I feel the old adage about regulations is going to be true here like it is with every new technology: The rules are written in blood.

[0] therh lasdkjenail EMAil was sposmkehting alLIKE HTIS IN THE WAYS THAT IS aw adklhjf writtene sand htenn. sned t sent ASLAOSNG ASLIONG ALONG TO NTHE recipentnts .

Sorry, but I highly doubt that. Has a very "old man yells at clouds" vibe.

Not taking the SAT doesn't explain a jump from 10% F's in 2025 to 35% F's in 2026.

It's the universities that have failed. They've restricted admissions to a set of people who would learn no matter what the schools did, which is what makes them lazy.

When confronted with a set of students who haven't been provided with an enormous amount of childhood reading material, and the time, encouragement and social acceptance to indulge in it (the most faithful test predictor is childhood pleasure reading, the next best is parental income), they fail horribly.

The purpose of elite colleges for students is credentialism and networking, the purpose for the schools themselves is to force cultural conformity onto smart or extremely pressured students. They generally just tell you to go learn things by yourself. They have no particular insight into teaching, because they are supplied with students who don't need to be taught.

What could go wrong...

It reads as though you tried to use the quote to support your conclusion that "it's been a failure", but the quote and the original rationale are optimising for different things. Something can be a success in improving equal opportunity while still leading to worse grades.

Or to flip it around: we could say admission testing "has been a failure everywhere" because it biases admissions in favour of certain demographics. But that wouldn't really be a fair assessment because being free of demographic biases is not the purpose of admission testing!

Sounds more like the score is 3/3 (100%)

Would you have accepted them cooy-pasting code from libraries together to build their project? If not, why is using LLM generated code different?

In the industry, almost everyone here is doing the exact same thing, outsourcing more and more of their thinking to LLM: so even if students learned how to manually write code, they will probably loose it later on (happening to us already, mostly can be seen when people are working on new solutions/frameworks... they now have the same issue than students).

It's cultural evolution and it's how markets work, too. You were expecting central planning?

And what possible benefit would that have?

On the plus side, high grade + long ago remains a signal.

Where are the incentives at a societal level to prioritize quality over efficiency? Of course if everyone did things the "correct" way we'd have no issues. But societies incentives are messy and contradictory, so taking shortcuts ends up being the more appealing choice in the moment for lots of people.

This is as much a societal issue as it a technological one.

rather than perceiving AI as a danger you should be looking at how he can leverage it to accelerate and enhance his learning but the political environment focus on removing standardized testing to hide data of those that traditionally fail is the true danger.

Sounds like you would hard-ban your son from using Internet if it was only introduced 5 years ago

Are you maybe saying that "soars" might mean "get better", so "failing grades soar" might mean there are actually less failing grades? That's not how I've ever understood that word.

Anyone with a pulse can declare a CS concentration at Harvard and muddle by (you actually need to try in order to get a C/C-). Of course, GPAs are calculated differently at Harvard compared to other universities, as a B- is treated at a 2.67 but most other programs treat that as a C+.

People can use AI to outsource their learning, but if they use ai to outsource their understanding they just set themselves up to fail even more.

From what I’ve seen, how students are using ai (not that they are using ai) is making them less prepared for the real world, which unfortunately is changing faster than ever at the same time to create double impact.

Ironically, the techniques of the latter yield the results of the first, but everybody gets to keep a pure heart.

Sure we could use our brain power with old techniques to do these, but why? I don't want to do any of these. I'd rather use that brain power for other problems.

Same with maps.

I don't want to have to store a bunch of location or routing data in my head.

I think what you're pointing towards is going from having problems to solve to not having any problems to solve.

That's definitely a danger, but right now is still early in the AI era so obviously it'll feel like we went from solving problems to letting the new tool solve them for us.

There are still many problems to solve.

I had classes where I didn’t make over a 50% on any test and still got an A because half the class dropped and the other half hung on for the curve like I did.

I think curves are more a result of poor teachers than anything.

Where I'm from (Norway), the majority of computer science and software engineering studies do not have the same math requirements as, say, engineering or math/physics/etc. - nor do they have the same amount of math as the latter ones.

When I did my CS classes as an engineering student, I did meet a bunch of students that viewed math as some niche subject only relevant to those that wanted to work with computer graphics, computational stuff, or similar.

Not because the actual truth encoded in it would be this complex, but because the encoding scheme just sucks.

I see it as a packaging problem that has so far not been painful enough to trigger any meaningful change.

With this LLM-driven collapse, that might finally change.

Idk I'm hopeful.

Math is literally the law of the universe. It makes zero sense that the way that it is taught needs some special brain wiring only found in small chunks of the population to truly click.

Understanding math well might help a bit, but they're the least mathy classes in the core Berkeley CS curriculum IMO.

If you’re doing this than great! More power to you. But I think you underestimate the discipline it can take to focus on a task when “coding is free”. I don’t worry that I’ll just lose my old skills. I worry that in losing my old skills I’ll lose the new skills as well. Driving is great! But I still run for exercise and occasionally to get places too. I’d be a worse driver if I was out of shape do to the loss of energy and mental clarity.

Do you have evidence that it ever was part of being a competent mathematician? AIUI the trope of mathematicians who can't even do arithmetic was common already before the pocket calculator was introduced last century.

Sometimes I don’t wonder if this wouldn’t still be a good way to educate people. Part of the problem is education has to sort of optimize to try to educate like passive people. If you’re a curious and pragmatic person, you can understand how to use what you learned in a liberal arts degree to be better at almost any job.

As I look forward to the second half of my career. Certainly I use AI in healthy doses.

But people talk about the division between practice and performance, and most of my practice is old school. Reading books. Writing my thoughts down. Memorizing quotes and passages.

I think more important than what you learn is the way you use it to train and evolve your brain, with the caveat that - I know this is more useful to me because I have a marketable skill. This is the balance universities have to stick, there are tons of people with liberal arts degrees in middling jobs.

But at least half if not more of education should be on building practical skills in the three r’s.(maybe the third r should be ‘rgumentation instead of ‘rithmatic, but I digress)

It’s interesting - people decry memorization in education, and I’m not entirely naive as to why - if you were to show up to the first day of work and say “I don’t know any of what you just said but I can recite log tables! It might be your last day - and yet one of the most underrated skills, especially late in your career is the ability to ingest and operate in large quantities of information.

That would be closer to engineering or accounting than mathematics. I don't think mathematicians do much arithmetic at all.

This seems like the crux of the issue. Like people are banking on that day coming even if they don't know exactly when.

When you're up against a deadline - and unless you're very good at time management you're frequently up against a deadline - it's going to be an irresistible lever to pull.

In times past, cheating would mean copying an answer off the Internet or off a friend, both of which are easy to detect. More sophisticated cheaters might spend an hour rewriting the solution to make it less obvious they cheated, but at some point the cost of cheating (time + risk of getting caught) starts exceeding the cost of just doing the assignment. AI changes this - you get a customized answer that doesn't show up in a database with no extra work.

The thing is, students fail to realize just what using AI robs them of. Struggling with the assignment is the entire point. You don't learn if the assignments are too easy; you need to have some challenge to push your brain to understand the material more deeply and to build those pathways to apply the knowledge in novel ways. You become more efficient and effective over time as that knowledge settles in and you get more proficient - one of the reasons why time-bounded exams still make sense (being fast is also a proxy measure for understanding).

Of course many people in a competitive environment will click the autosolve button if available. This is a reason to think how to redesign the system so that the approach we want is the reasonable choice, not to look with superiority at those who fall prey to the danger.

Also I really question what this litmus test would check for and how accurate it is. Just the other day I read an article that frontier AI models outperform law students and that professors could not distinguish it from a human written ones and actually flagged human written ones as AI.

I do view this as overall positive progress in that the ridiculous barrier to entry and wasteful formality is being eroded and as will the professions that benefited from that moat will open it up to the general masses.

I personally believe we should be celebrating the universal access to knowledge and printed words

There are several reasons for this:

1. Cheating in CS is easier to detect. MOSS [2] (authored by CS professor Alex Aiken) is a very effective tool at detecting plagiarism in coding assignments. Personally I witnessed more honor-code violations in math problem sets, but there was no feasible way for professors to detect this.

2. Problems in programming assignments are (usually) very tangibly wrong. I can bullshit my way through an essay with shoddy research, I can hand-wave a proof that is definitely wrong but will probably garner at least some points. But when your program is crashing or not compiling, and the due date is approaching, it produces a very immediate and undeniable sense of failure and pressure to cheat. The thing is, many students would get a decent chunk of credit even for failing code, but this is not immediately obvious.

3. The ability to cheat is more available. Math problem sets tend to change quarter by quarter. It's basically impossible to cheat on a prose essay short of straight up paying someone to write it for you, or fabricating sources. But for CS classes, especially at prominent universities, there are plenty of solutions online. Much of it is people who aren't event at Stanford implementing the assignments for fun or self-learning, and sharing it with their peers. Which, to be clear, isn't unethical or bad - it's the responsibility of Stanford students to refrain from looking at those solutions. But nonetheless, it's a contributing factor.

1. https://stanforddaily.com/2015/03/29/increase-in-cs-106-hono...

2. https://theory.stanford.edu/~aiken/moss/

The whole situation sucks for both students and teachers. Teachers know that the knowledge they're going to great effort to convey isn't going anywhere. Or at least, it's landing in far fewer fertile brains than it used to. Students are squeezed because part of the university experience is being forced to adapt to an academic load, and as a result change yourself in ways that benefit you (or at least produce learning!) There have always been relief valves -- not just forms of cheating, but blowing off a study session by using game theory on your grade or going to a tutor or taking easier classes or extending your stay at the school. But now there's this huge giant relief valve in the form of a shiny LLM that is always available, particular at 3:45am when your project -- the one you've steadfastly refused to use AI on thus far -- is due the next day. The schools have tuned the pressure for the old set of options, and it's not clear that there's a new tuning that maintains anywhere near the old level of learning.

I guess my question is: of those students who were flunked for cheating, how many of them were learning despite their cheating? (And how about the students who were cheating but not caught?) Also, what levers are there to move more students towards learning even with the chatbots present?

I'm sure these questions are being debated. I know Garcia personally, and he is very invested in his students learning. The title of his Joy course is legit. So I'm sure the profs have ideas around this, though clearly not happy ones. Perhaps I'll ask him.

> The solution? I'm not sure

This initially felt like you were setting up a joke. If you feel like something is harmful to you, stop doing it. Find alternatives (they are there, it’s everything else; commercial LLMs are still fairly recent). Thinking “maybe I don’t have to let it go, I can still use it if I do it this other way” sounds like an addict justifying themselves.

I say all this without a hint of judgment. I genuinely hope you are able to tackle the harm you’re feeling.

The solution is extremely obvious, just stop using it on 2 days out of the week or something like that.

You need to go to the gym, but for your brain.

If what you are building is too complex for you to meaningfully contribute to in the absence of LLM assistance then that should tell you something important.

Plagiarism isn't new, and those things enabled it too.

Set a reasonable bar for grades or SAT scores and then use other criteria beyond that gate.

"Enlightenment is man's emergence from his self-imposed nonage. Nonage is the inability to use one's own understanding without another's guidance."

https://www.columbia.edu/acis/ets/CCREAD/etscc/kant.html

I would grant: I was not the most studious kid, I could definitely stand to learn how to read code a lot more effectively than I do; but I have found being able to ask a computer, "what portions of the Vulkan Programming Guide are less relevant with Vulkan's design changes since the release" pointing me to the dynamic rendering extensions and placing it into context, with inline code and links out to useful blog posts for additional reading, that sort of thing is very helpful.

Working on a prototype before I was trying to learn Vulkan, I was using it to explore SDL_GPU's API which definitely had some gaps in its documentation. Granted again, I could have referenced the sample code - I am sure you'll prefer I'd have done that - but it helped to get information about what each piece of the API was doing, and gave reasonable results that made sense and did inform me enough to understand what I was doing, turning much of that into an interactive learning of basic GPU programming for graphics. Where the AI hallucinated, it was often on things like method names, which I was able to read through and find the methods it was intending to name. (This only occurred once or twice when I was learning).

Unrelated, but adding the C macro syntax and nesting macros, which I could have an LLM explain inline and link the GNU manual. Never got that taught to me in a C course. Man, computers are complicated!

These have not replaced textbooks; I have been using them alongside textbooks and handwriting code for practice, and they work as a very good complement. I also sometimes use them to unblock me - I don't know CMake very well and lean on AI to do CMake, so I can focus on learning C++ and graphics, which is my primary objective right now.

I would add too, I have for fun given it prompts about various topics I learned in university, and I often will get answers that are bang-on what I learned in university undergraduate courses - the topics I tried were welfare state taxonomies, distributed systems, disk storage performance, filesystem layouts and internals.

Boy, this would've been cool for me as a kid. There's just so much information right there, and pointing you to topics and textbooks a couple questions away, I wish I had these tools. I was a curious kid in a terrible MAGA-esque family that was deeply uncurious about the world, had no knowledge of any advanced subject and basically mocked me for trying to learn more about stuff. And you go to the school library and it's all kids shit, not even an option to try and reach out for more. Now smart kids might be able to go just learn shit very freely and be pointed to textbooks, and go pirate them off some Russian site, and start learning and go tutor themselves, as I'm doing today as an adult.

At least knowing myself and knowing if there's another kid like me, I think they would deeply enjoy having a natural language encyclopedia, if we can get it as close to that as possible. I think even with some error inherent, if the tools can be often and directionally correct, that would be a plus. I went to university, and the professors there hallucinated some things so embarrassing it should bar them from teaching, for the standards people hold LLMs to! i.e., sanitizing conspiracy theories that Android records all language through the microphone therefore iOS is better, Apple Silicon is more battery efficient because it is RISC and not CISC. Got a terrible history of computer graphics technology you'd know was slanted if you watch the 8 Bit Guy on YouTube. Rubbish.

The thing that worries me, and what this article really talks about, are the kids that just don't give a shit. They are not new - when I went to high school, before AI, stupid kids would copy code off the internet. I think AI probably makes it worse because it makes it harder to call out and enforce against it, and agreed, that should be stopped. But to me, that is mainly a cultural problem. Too many Americans are completely uncurious and just spout garbage; there are a lot of kids who grow up in that cesspool and are going to grow up uncurious, and then AI acts as a shortcut rather than a vehicle of curiosity.

And granted, maybe AI is less useful when you are in a structured environment - but the structured environment has its downsides. Even in that environment many of the TAs were clueless and unhelpful, or just too damn busy or already too knowledgeable to meet students where they were at. Again, talk about hallucinations with TAs! Many times in my experience. And that's all to say nothing about getting people to not just do homework but actually go get curious about things and try stuff that isn't required of them.

I think there will be some culture that remains curious, and has these tools, will come to grips with where they can help, where they go wrong, how to balance it with other learning methods; and I think they are going to have kids that absorb a lot more knowledge and get to play with topics and learn things, faster, to each kids' interest, perhaps even individualized tutoring at better scale - I hope that is possible.

I hope the United States as well, but maybe not, because holy cow our culture and attitudes are plainly terrible these days. Your comment is pretty representative of how most people react if I suggest this or talk about my own experiences I'm describing here. But I hope at least I'm arguing something comprehensive here. There is too little conversation beyond hyperbolic nonsense on the internet; I consider "FUTURE LUDDITE" etc. to be in that realm.