back
55 comments
> There's plenty of things wrong with these tools: they are often wrong, are slow and the GPT4 ones are really expensive.

So...why should they be included?

I really worry about this "often wrong" part - you only know they are wrong if you already know what you're doing. Otherwise you end up trying to use hallucinated APIs & libraries, or produce code not better than copy & pasting StackOverflow answers (which is what the AI was trained on anyway).

If an LLM hallucinates a method name you'll find out the moment you try and run the code.

Code has a built-in form of easy fact checking, which makes it one of the most appropriate applications for LLMs. It's much harder to spot a hallucinated fact in a paragraph of prose than it is to spot a hallucinated API method.

The skills you most need to develop in order to take advantage of LLM assisted programming are code reading, code review, manual and automated testing and being really good at thinking of edge-cases that might not be covered.

It turns out these are important skills for being a great developer already - LLMs just force the issue on them a little more.

Over reliance is a major part of the issues. LLMs in other contexts help bottom performers a lot, but top performers get worse because they are lulled into trusting the AI's judgment.

I've worked a good number of hours with Claude Opus and it has never produced non-compiling code (ChatGPT 4 does that for me), but it can create quite subtle bugs, which is missed by the "just make sure it compiles"-type comments in this thread.

This is like not using keyboards because of typos.
> Note that this is often not the same as "#9 - Do you use the best tools money can buy?" as blocking AI tools is about data security, not money.

I do think AI fits #9. The fact that current AI tools are not meeting data security requirements are due to the market demands and maturity:

- price needs to be low to attract adopters.

- low price? These service providers will hoard data

- data needs to be collected for training

So i think long term, there will be more premium AI tools that “promise” to not collect your data. Perhaps self-hosted? Self hosting with AI is not attractive, at least not for consumers or small businesses.

OpenAI have paid plans that promise not to collect your data already. I think Anthropic do as well.

People seem not to trust companies which make these promises, which is unfortunate for the industry.

Self-hosted is becoming more and more possible. For the first time in forever I think it makes sense to buy a beefy personal computer.
I don't remember the Joel test being about "add the latest hype to your dev environment"
One quibble I have with this: the items on the Joel Test are all language-independent, even if you're using a minor language with poor presence on GitHub. But that's not the case with AI codegen. GPT-3.5 was downright terrible at F#, and though Bing+GPT-4 does seem a lot better I'm not sure it's actually good enough for professional F#. I also doubt AI codegen is useful for Scheme/Racket developers who create DSLs (and of course it's totally useless for proprietary languages). So I am not sure AI codegen fits with the other items on the test.

In general I am concerned that LLMs will discourage innovation in programming language design - why write a better Python if GPT can just automate the tedium away?

There is a code-gen feature in a product I know of. The product has a proprietary language and the user base clamors for AI generation. Part of the suggestion process in the tool is to feed generated suggestions through the parser and a basic semantic validation before putting the suggestion in front of users. There is still no guarantee of correctness, but the code will parse and run thanks to this. They have to implement some limitations, because it becomes expensive to re-generate repeatedly.

I believe the knowledge I gained was NDA, so I'm keeping it generic.

> I am concerned that LLMs will discourage innovation in programming language design - why write a better Python if GPT can just automate the tedium away?

I don't think so negatively. My bet: innovation in programming language design will emerge that will make it a lot "less necessary/helpful" to use AIs.

Just one example: Quite some programmers claim that AI take a lot of "tedium" from the programming away. But what if we could create programming languages that mostly get rid of this "tedium" (e.g. by using higher-level abstractions to abstract away the tedious, repetitive tasks)?

> blocking AI tools is about data security, not money.

Why is AI treated very different than say cloud? Most companies don't have problem with putting all data in Github or AWS or Office 365, but lot of them freaks out if any AI can access the data. I don't think OpenAI/copilot enterprise plan T&C/privacy policy is very different than Github or AWS.

I think it has less to do with the fact of putting data somewhere, but with the way data is accessed. When you store something in S3 - you can encrypt it, in addition to built-in encryption. You have control (more or less) over your data in your cloud tenants and databases. Not with AI models trained on your data. You can't even extract\remove it or see how it's used. Literally - zero traceability and transparency. This is the problem, not the fact that it's not stored on your physical hardware.

I suspect when any AI model will start using patents databases for training - it will be a watershed moment for what one can do with open data. Old regulations simply would not put up any meaningful fight against volume and quality of model hallucinations, that may become valuable and patentable inventions and improvements according to the same regulations.

People freak out when their data is used to train AI models, leaking their private data. AI data is regularly mined, whereas cloud data is usually encrypted in transit and at rest.
One of the first things we did when GPT-4 became available was talk to our Azure rep and get access to the OpenAI models that they'd partnered with Microsoft to host in Azure. Now, we have our own private, not-datamined (so they claim, contractually) API endpoint and we use an OpenAI integration in VS Code[1] to connect to, allowing anyone in the company to use it to help them code.

I also spun up an internal chat UI[2] to replace ChatGPT so people can feel comfortable discussing proprietary data with the LLM endpoint.

The only thing that would make it more secure would be running inference engines internally, but I wouldn't have access to as good of models, and I'd need a _lot_ of hardware to match the speeds.

[1] - https://marketplace.visualstudio.com/items?itemName=AndrewBu...

[2] - https://github.com/mckaywrigley/chatbot-ui (legacy branch)

I must admit that Copilot usefulness for me is on the level of IDE autocomplete. I can write code in notepad.exe if necessary, but that's not productive and getting productivity hit is not nice for wellness.

So if company will not allow me to use Copilot, that would be a negative factor from me.

Bing with Copilot is insanely good and initially I was having a tough time getting my slower teammates to adopt it for the more tedious tasks.

I don’t really use it to do anything but tedious stuff and for searching for documentation that google will refuse to show anymore. and it provides sources so you can verify. It really does feel (to me) like the magic of google search’s majestic era, like 2010-2015. It just tends to give the correct answers at an extraordinarily high rate and can be poked and prodded in the right direction without a lot of work.

The problem is not writing code. The problem is writing the right code and supporting it as the product evolves. AI tools result in programmers that know less about their code and therefore less able to support and less able to contribute to the direction of the product. Notice that almost all of the Joel test is about how to support code over time, not which editor to use.
Trying Codeium was like the difference between notepad and an IDE.

Unfortunately, it overheats my laptop so I can't actually use it, and I primarily do support, I don't program enough at my job to justify paying for copilot.

If the CPU use was lower, I don't see why I'd ever go without.

I agree, but they should count negatively.
Yeah I've thought this myself for a while; GPT-4 class assistance is now a standard tool for me.

If a prospect client or company bans it, it's a hard no from me.

I understand that might be too extreme a red line for some, but for me, life's too short to wait for laggards to catch up with the inevitable.