Item — HN Reader

Bing Chat (Choose "Creative Mode" or "Use GPT-4" depending on UI) is GPT-4 with retrieval augmented generation using the Bing Search engine. It seems to be tuned a little differently, but I haven't found it any better or worse than GPT-4 with browsing enabled.

Note that if you don't pick "Creative Mode" or "Use GPT-4, you get Microsoft's own LLM.

There is a new GPT-4 Turbo available from OpenAI that is ahead of GPT-4 on many benchmarks:

https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

My guess it that it will soon be incorporated into Bing Chat. So if money is tight, I would stick with Bing Chat (with the appropriate mode turned on). Note that Bing Chat is very different from vanilla Bing Search.

Bard and GPT 3.5 are decidedly inferior to GPT-4. I wouldn't waste my time using them.

Github Copilot is also markedly inferior to GPT-4 in generating code based on instructions.

Sometimes I get a little impatient waiting for Bing Chat to generate an answer. If the question is not too complicated, I have found perplexity.ai (w/o GPT-4) to be low latency and high factuality. In fact when searching for research papers on a topic, I have often found it superior to the alternatives.

3 comments

Not sure if you've used Bing Chat recently. It's performing worse and worse over time. One pattern that I keep running into in almost every conversation is that it doesn't acknowledge the question and just answers something else. When I clarify myself and ask for a better answer it starts repeating itself verbatim, again and again, and there's no way to get it out of this.

Additionally Bing Chat has a limit of messages per chat, which means you lose your context over and over again, which is a significant impediment when context is important to the problem at hand.

Another problem is Bing Chat is much more likely to search online, than ChatGPT 4. ChatGPT searches when this is logically necessary, like asking for latest news. Bing almost always searches, and then IGNORES its own knowledge, in order to give out a poor digest of what it read online. The problem is that what it read online is often not relevant to the question, because it depends on how accurate the search query was. And if I wanted just to read the search query I'd go search myself.

Yes, I have been using it the last few days. I haven't noticed the failure case of Bing Chat not answering the question at all. Are you sure you had the modes I recommended turned on? Try reporting the problems to Mikhail Parakhin, the Microsoft CEO of Advertising & Web Services, on Twitter. Most of his messages are about bugs and features in Bing Chat and generative AI products.

https://twitter.com/mparakhin?lang=en

Definitely agree that Bing Chat's message limit per conversation needs to be kept in mind. I haven't found that to be a problem in practice, since I start a new topic with the most relevant recent context. OpenAI's GPT-4 seems to be doing some smart summarization behind the scenes since you can keep going.

I agree that there are occasions when the search results confuse Bing Chat, and GPT-4 (w/o browsing) performs better.

Your thoughts about where GPT-4 excels over Bing Chat are quite insightful! It would be great to see some examples prompts/questions where you feel that the quality has degraded over time. I am sure Parakhin would be interested as well.

Thank you for the pointers. Frankly I mix up the modes a lot usually, trying to get a good answer, and since it's been a few weeks (I kind of got annoyed and stuck to ChatGPT-4 from then on) and I can't immediately recall if Creative Mode does that specifically. My memory is all modes do, but that may be wrong. So I'll follow your advice and also noted who to ping on Twitter, cheers :)

If you were logged into Bing, those prompts may be in your history. They can be viewed using the Edge browser.

A few weeks ago, I had spotty service with Bing Chat where it would keep resetting the conversation which I assumed was due to load. In general all these LLM services are in constant flux because they are tuning both the models and UI. They feel like alpha quality products in terms of stability.

>Github Copilot is also markedly inferior to GPT-4 in generating code based on instructions.

Is that still true? I noted a significant improvement with the chat function. Especially the ability to mark sections of code for review/discussion is something that you cannot easily do with ChatGPT4.

Github Copilot Chat is in beta and is marketed as a separate product. I suspect it is using a later generation (and better) underlying model. I used it many months ago, and I agree it is much better than vanilla Github Copilot. I haven't done a careful comparison to GPT-4, however. As you point out, the integration with the IDE does save a lot of time. Developers should definitely try it out.

https://docs.github.com/en/copilot/github-copilot-chat/about...

I cannot pay even if I want to. :( There is a waitlist. Are people just using OpenAI APIs? or chatgpt+?