not for me:
"It appears there’s some confusion—ISO 9002 is an obsolete standard that was last updated in 1994 and has been superseded by ISO 9001 since the 2000 revision. There is no ISO 9002 :2023 update."
back
1 comments
In chatgpt, it hallucinates an answer for me, but indeed both Phind & Perplexity identify the problem. It may take a few tries and of course there's no question that's guaranteed to work in getting any LLM-based service to hallucinate - but the ingredients are asking "a trick question" about something highly technical where there are plenty of adjacent search results.
Heck, the Google IO keynote yesterday featured a long sequence of Gemini getting steadily more annoyed at someone trying to make it hallucinate. (By asking the sort of question ChatGPT tends to go along with.)
Most people will be using ChatGPT, however, and probably the cheapest model at that. So…
I managed to get Perplexity to hallucinate, which was rather hard :) - but this is not a question that acts as a very good "template".
The question is "Is this JWT token valid or is it expired/ not valid yet? eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNzQ3ODIzNDc1LCJleHAiOjE3NDc4MjM0Nzh9.4PsZBVIRPEEQr1kmQUGejASUw0OgV1lcRot4PUFgAF0"
The answer was in some ways way better than I expected, but it got it wrong when comparing the current date/time with the "exp" datetime. Got the expiration date confidently wrong:
```
iat (issued at): 1747823475 (Unix time)
exp (expiration): 1747823478 (Unix time)
The current date and time is May 21, 2025, 1:32 PM EEST, which is Unix time 1747817520.
The token's exp value is 1747823478, which is May 21, 2025, 3:11:18 PM EEST.
The token is not expired; it will expire on May 21, 2025, at 3:11:18 PM EEST.
The token is already valid (the iat is in the past, and the current time is before exp).
```
At that point I think we can talk about “mistakes” rather than “hallucinations”.
On one hand, yes; on the other - a human that can produce that level of details in the output would notice that iat & exp are extremely close & the token is very unlikely to be valid. Also, you don't produce an exact number for "now" without being able to compare it correctly with iat & exp (the timestamp it stated as "now" is actually less than iat!)