back
111 comments
I didn't scour the text but one thing I didn't see as an explanatory consideration in the paper is that the larger companies they talk about having larger adoption are also much more likely to have reasonably robust controls (and consequences) that prevent mixing personal/corporate use of AI tooling. That's my world.

On the other hand my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement. She also uses it to help with analysis of testing data against her lesson plans and state standards to find places to shore things up as the year progresses. She runs the english language learning department in her school and also uses it to help ensure that there is representation in her materials for the huge mix of cultural backgrounds of her students.

It's truly helped her immensely and it's all been on her personal account. The district has no program or budget to guide or support teachers and admin on using these tools in education.

Such a shame.

Schools for the poorer segments of society keep adding more and more technology, eroding the analytical skills of their students and staff. But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child. This keeps the administrators happy.

Meanwhile ultra-expensive elite private schools for the children of the Zuckerbergs, Al Sauds, Al Nahyans, where the teachers are Ivy and Oxbridge graduates emphasise tradition, classical studies and analytical skills while relegating technology to a minor role

As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...

This was more than a decade ago when we were looking into emersion schools for our daughter. We went to several private schools first and all three were emphasizing ipads as their primary tool for teaching language. The public school our daughter was set to attend had an open house promoting its long running emersion programs.

Wow. Our public school had native speakers and no ipads. There was a major focus on true emersion. From minute one, they were spoken to in the native language. The teachers were incredible. In the first six months, my daughter was speaking in full sentences and her teachers were praising her pronunciation of certain words.

The real eye opener came at a local park. My daughter was playing with a boy, speaking to him and his mother. I came over to join the fun and the mother started speaking to me in Spanish. I was like, "Uh, I don't speak Spanish." She looked at me confused and said, "Your daughter, she was speaking fluently to both myself and my son in Spanish."

It was a pretty jaw dropping moment. One, to think we were blessed to have such a great emersion program at our public school and secondly to see my daughter pick up a language so effortlessly. It really reinforced my idea that its all about the quality of education, not necessarily how much you're paying for it.

> But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child.

Baltimore spends more per student than Switzerland.

> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...

Education for more affluent people is often cheaper per student. The benefit richer students have is that, on average, their parents are smarter, more conscientious, etc.

Eh.

Prior to her current role my gf spent about a decade as the head of a very elite private elementary school (she's been in education for 35 years now). Certainly ten figure families represented in the student body with names you would recognize as such. The school had access to all of the resources they could ask for and they did very well on paper. However, the attitude of the students was qualitatively different (and in my gf's opinion much worse) than those of her current students.

It's fun to imagine a collection of rock star educators huddled around every child just constantly feeding their intellectual curiosity, but the reality is most of those kids just aren't hungry. They live lives of comfortable inconsequentiality which, on average, seems to rob something from human curiosity and drive.

This feels like conflating "schools use technology" with "schools outsource thinking to technology"
> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...

The rich and powerful pay "fuck you" money for their kids to get the best pre-college educations possible. It takes an incredible amount of willpower for a world-class teacher to say no to that type of money over even high-paying private schools.

Keeping down cost per child? What are you talking about? The cost per child has skyrocketed in the US over last 20 years, particularly among the worst-performing schools. Baltimore spends more per child than Mountain View. NYC spends more per child than Palo Alto.

There is a certain view, popular among many people, that poor performing schools perform badly because they are underfunded. It has been vigorously spread by politicians, teachers, and activists. Well, we ran the experiment. With enormous federal and state spending, we managed to mostly equalize the spending, and, in many cases, fund the badly performing schools more than the good ones. The results of the experiment are very clear: additional funding, by and large, does not *cause* significantly improved educational outcomes. Any observed correlation between school funding and outcomes is a result of who lives in the area, and who attends the school. The casual arrow does not go from funding to outcomes, but from the school population to both funding, and, in parallel, to outcomes.

I suspect one of the big advantages that money brings to the kids of the rich is to have not only one-on-one education, but also the best-of-the-best in each subject.

That said:

> classical studies

While this may be the "cornerstone of an elite higher education" as Wikipedia calls it, IMO (like studying Shakespeare[0]) I'm not convinced it is a productive use of time.

Don't get me wrong, nerd out about anything you like, but a lot of the world shifted since then. The overlap between what you learn from studying people who thought the sun was a chariot drawn over the sky by magic horses and that the Sahara was caused by some idiot driving them wrong, and what you need to know to deal with a highly connected globally integrated supply chain, is not exactly zero; but childhood is limited and you are likely better off focusing on what's actually in the modern world.

That said, of the few ways it is still useful is likely as shibboleth for the powerful to know one another.

[0] Shakespeare's closer to us than the ancient Greeks and Romans, of course, but still a very different world, the plot of e.g. Merchant of Venice just can't happen today. Plus he was seen as merely "good" rather than "the best"[1] within his own life, was clearly reworking other older tales for his audience[2], his plays were his era's version of somewhere between MCU, a soap opera, and stand up: the goal was bums on seats not high art, it was watched by a broad audience including normal people not just the era's version of evening-wear, crowd participation was a thing, actors would need to be reminded their lines in the middle, etc.

[1] https://en.wikipedia.org/wiki/Reputation_of_William_Shakespe...

[2] e.g. https://en.wikipedia.org/wiki/Sources_of_Hamlet, https://en.wikipedia.org/wiki/Amleth, why yes I do only know about Amleth from SMBC

> my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement.

My wife, who is also a teacher, does the same thing. I think this is an underrated use of AI right now. There's also been a new wave of edtech companies focusing on AI as the teacher, which I'm skeptical will reach product market fit. I think AI as the teacher assistant could be interesting though.

IMO, in everything, there will be a small cohort of incredibly motivated and dedicated to always giving their all for their job, which is also their passion in life. Then there are the people who at one point had that passion^, and are now doing the work to cash in a paycheck, and will take shortcuts where possible to replace the effort they would be putting in.

^ not that this is always something to blame people for. A passion to do something you love doesn't typically translate into doing it so that you enable your employer to profit.

Is your girlfriend secondary or primary, if you don't mind? Our secondary programs in the region have a lot of AI vendor and software money flowing into them yet like you I see them (the teachers, not the AI) crafting lesson plans and assorted work with free institutional accounts for these products or personal ones. The vendors seem focused on student acts; for the teachers, there's a lot of "say your lecture into this tool and it will provide feedback on your performance" type of thing, which seems specialized.
Primary ELL. Could certainly be the case in the secondary schools but yeah, nada for teachers, students or admin.
>She also uses it to help with analysis of testing data

>on her personal account

With all the PII stripped, I hope?

If orgs don’t want to pay for tools, that’s on them. Aka “Shadow IT.”
OpenAI is PII compliant so it's ok
I think a majority of my friends are teachers and this could not be further opposite their sentiments. They feel strongly that it’s dishonest, nearly akin to plagiarism to use AI output in lesson plans. They don’t use it for grading or data analysis b/c of privacy concerns. They mostly complain that the kids are using it for homework, which makes it hard to grade both because it’s too uniform and because there’s no good faith effort going into the work which makes reading it unrewarding. And the consensus is that test scores are down b/c they’re offloading normal course work to AI. It’s basically the bane of their existence.

The idea that it’s actually better for diversity feels like a wild reach, almost a canned talking point.

>The idea that it’s actually better for diversity feels like a wild reach, almost a canned talking point.

lol

She's got a photo of a new student from Cameroon pointing at a little laminated cartoon character that she hung on the bulletin board outside of her room.

It's a very effective way to make new students from far flung regions of the world feel seen.

How do you get to plagiarism from lesson plans? Or that it's 'dishonest' to use a tool to create them?

Genuinely not understanding the chain of reasoning there, unless you start from "Everything generated by AI is plagiarism by definition", which is objectively wrong and not based on how LLMs actually work.

Kids using it for homework is a different issue, and I would agree that's not good.

Lol ok tons of teachers use LLMs to make material. The plural of anecdote is not data, but in this case it's close.
Would it kill these OpenAI guys to learn how to write a white paper? Google's DeepMind did better with their AGI to ASI, though I have qualms on their execution.

Whenever I see these Frontier Labs "publish a paper" on ArXiv.org the phrase Esse Quam Videri strides past my mind—is this genuine progress or merely a marketing metric for their stakeholders?

you are complaining about a problem with this paper without explaining what it is.
When I say “how” I mean like how u/glouwbug mentioned: Formatting. But more so Structure.

When I read artifacts by Frontier Labs I notice a trend towards deteriorating lexicographic architecture. Like how a textbook or larger tomb is constructed for the reader to navigate.

Instead, now I see continuous monotonic text. It’s as if the author expects us to chuck the artifact into an LLM in order for it to inform us of meaning.

This is a typical working paper (preprint) not white paper, for its field. You can wait for the journal publication.
"badly". There, fixed it for you.

Seriously, the best blog post on this subject has to be Ludicity: https://ludic.mataroa.blog/blog/ai-mania-is-eviscerating-glo...

Purely anecdotally, over the last few weeks I've noticed a significant uptick in articles trying to convince me how awesome AI is doing. And that's from a pretty high baseline of articles that were already telling me how I should really appreciate how awesome AI is.

It's almost as though there's some worry that the wheels are going to come off before the right people are able to cash out.

isnt it how marketing is done these days? people/companies buy views, clicks, articles, posts on social media saying how great something/someone is or how its changed their life etc etc.

it really becomes hard to separate the wheat/chaff for alot of these things...

Thanks for the article! Too opinionated imp, but an interesting read.
It feels like Im in a different world than this author. Cluade code is now more reliable and trustworthy than most of my coworkers but this author hasnt seen a single AI initiative succeed? Seems like theyre just being obtuse because the only other explanation I can think of is user error.
I think there's a huge difference between someone using AI to help them build something, and a large organisation attempting to become "AI Native", whatever that means.

We saw the same during the dot com boom - lots of companies trying to become "dot coms" because that did good things to their share price and they could answer investor questions, but no-one actually in the org knew wtf they were supposed to do to make the company an actual dot com. Trying to define what a "dot com" actually was, was difficult enough. Then the bubble collapsed and investor pressure on change relaxed, and a quarter of a century later we're pretty relaxed that some companies exist almost solely on the internet, and others don't. Both for reasons that mostly make sense.

It'll be the same with AI - the bubble will collapse, the pressure will relax, and we'll go on to have some companies almost entirely built out of AI, and others not.

But the poor management folks who have to come up with some answer about how to make their organisation "AI native" immediately - you could not pay me enough to do that job.

This reads like some crypto or NFT commodity white paper, i.e. like an ad that says you need to use this now, and more, so that you'll figure out what it's good for.

A difference is that in those white papers they usually argue that their solution is more perfect in some sense than some more adopted alternative, here they instead claim that their commercial offering is "general purpose".

I get the ad-like vibe, but I think that's slightly unfair to the paper. "General purpose technology" is being used as an economics term here, not as a claim that ChatGPT is universally useful
They draw a random sample of matched adopters but use the full Compustat population of US companies as non-adopters and don't disclose the sampling rate, which is statistically risky.

Also doesn't account for pilots. If a Enterprise customer has 10 licenses for a pilot, they could be in there.

Interesting data, but I think there's a pretty big trap in treating usage intensity as a proxy for economic impact. The finding that early-career workers send substantially more messages than senior people could mean they're getting huge productivity gains. It could also mean they're the ones doing the work that's easiest to delegate to ChatGPT…
Conclusion: no measureable ROI. In fact, the enterprises have no idea where or how to begin measuring (we don't either tbh). We are still early. Keep buying our product, so we can find out, together.
What is difficult about measuring RoI? Time scales?
Question: "How"

Answer: Yes.

Answer: differently and increasingly
69 pages long.
I think you're supposed to use AI to digest it
The main text is only 23 pages double-spaced. The remaining manuscript comprises images, tables, and references. It's not that long really.
nice