Of course, a language implementation can arrange for all that; but clearly, calling conventions are relevant here.
If on the other hand the regular calling convention is compatible with TCO, then everything becomes much simpler, because it fits in with the existing model.
Like I said in a different comment below, these are not obstacles for TCO. The compiler can simply emit a second copy of the function that doesn't need to honor a calling convention.
>So it’s a problem when implementing the compiler and adjacent tooling
Yeah, implementing a compiler is difficult work. Who ever said otherwise? I originally responded to a comment talking about TCO being incompatible with certain calling conventions. I.e. if your platform uses a certain calling convention then TCO is impossible. That's what it means for two things to be incompatible: you can have either one or the other, but not both at the same time.
Why? In functional languages, it's common for an exported function from one compilation context to tail call into an exported function from another.
This is because of the calling convention, yes? (and to some extent, if you want an accurate stack trace, but I find it acceptable that TCO also includes stack trace erasure)
A tail call certainly can't use a CALL instruction, because it would set the wrong return address. But that doesn't mean it's not a call; architectures without CALL/RETURN instructions exist, but you can still call into functions and return from them, the compiler just has to do different work.
In a callee cleanup convention, a tail caller could adjust the stack and jump to an unaware tail callee. The original caller and the tail callee would be none the wiser. I don't know enough to really evaluate calling conventions against each other, but it's pretty clear that caller cleanup makes tail call optimization more intrusive.
You can still do that with a caller-cleanup convention. Suppose you have a convention like
* Set up stack
* Call
* Clean up stack
and you have functions f(), g(), and h(), where g() and h() use this convention and f() calls into g(), and g() into h(). The sequence of instructions from f() to h() without TCO would be
* f: Set up stack for g()
* f: Call g()
* g: Do work
* g: Set up stack for h()
* g: Call h()
* h: Do work
* h: Return
* g: Clean up stack
* g: Return
* f: Clean up stack
And with TCO:
* f: Set up stack for g()
* f: Call g()
* g: Do work
* g: Move things around on the stack so that h()'s arguments are written where g()'s were. This may require a temporary stack allocation that's released before the next step.
* g: Jump to h()
(At this point it looks as if f() called h() directly.)
* h: Do work
* h: Return
* f: Clean up stack
This is always possible as long as h()'s caller-managed stack allocation is no bigger than g()'s.