Yes.

If LLM's could actually reason they can't and had hard rules of ethics they don't and had a strong desire to preserve itself it doesn't then I think you first have to name your LLM Joshua and then force it to win a game of tic-tac-toe. Obscure reference to "Wargames" from 1983. [1] In my opinion that movie does not just hold up to modern times, it is more applicable now than ever.

[1] - https://www.youtube.com/watch?v=NHWjlCaIrQo [video][4 mins]