To summarize at least one of them, the compiler doesn't really see it as “detecting undefined behavior and optimizing accordingly”. It sees it as doing the right thing for all defined behaviors. The sort of imprecise analysis it does lead it to consider plenty of possible undefined behaviors, many of which cannot happen in real executions. It ignores these as a matter of fact, but reporting them would not tell the programmer anything it doesn't know, and would be perceived as noise.
On the example for (int i=1; i==0; i++) …, the compiler does not infer that i eventually overflows (undefined behavior). It infers that i is always positive, and thus that the condition is always false.
Besides, fixing spammy warnings shouldn't be more difficult than fixing actual spam! I mean, common, with spam you have intelligent adversaries and compilers haven't reached that level. Not yet, anyway...
For a trivial NULL-related example, the standard C free() function does a NULL check on its parameter. free(NULL) is legal and does nothing. A naive "does this check for NULL after dereferencing the pointer?" checker would therefore warn for this code:
printf("the pointer's value is %d", *p);
free(p);
To a human, this obviously shouldn't be warned about, while the other example should be. But how does the computer tell them apart? It's hard.But! You made a good point, and it would apply to a function that did a null-check which was inlined. It's easy for us to imagine a function which 1) does a null-check, 2) gets inlined, and 3) is used in places in the code which dereference the pointer before calling the function.
Issuing an error or warning about this would flood stuff with warnings due to inlining/macros, you name it.
This happens all the time.
Basically, distinguishing between the things that are accidents, and things that are on purpose and expected to be optimized away, is very very very hard.
If you know that p is not NULL, the code is just fine. The compiler has to compile it.
Nick Lewycky submitted this code:
#include <stdio.h>
#include <stdlib.h>
int main() {
int *p = (int*)malloc(sizeof(int));
int *q = (int*)realloc(p, sizeof(int));
*p = 1;
*q = 2;
if (p == q)
printf("%d %d\n", *p, *q);
}
This got my attention for a lot longer than the OP, because it maintains the surprising behavior (prints different values for * p and * q even if p == q) if you move the assignments inside the if-statement: http://codepad.org/PBUAgnQqI'm told that a pointer passed to realloc has to be assumed to be invalidated, even if it's exactly equal to another pointer that you know is valid, but it's hard to wrap my head around that and I certainly didn't get that out of looking at the C89 standard.
If the compiler is indeed allowed to assume that the pointer passed to realloc() becomes invalid, then I would expect it to actually optimize out that entire if-check, under the assumption that the `*p` is undefined behavior, and therefore that `p == q` must never be true.
Getting different values for the print if you move the assignments inside the if statement suggests to me that a) it's assuming the pointers don't alias, and therefore b) that it assumes it doesn't have to read the values back out of the pointer when printing them but can just reuse the values it knows it wrote to the pointer. But if it assumes the pointers don't alias, then I would think it would assume that means `p == q` can't be true.
#include <stdio.h>
#include <stdlib.h>
int main() {
int *p = (int*)malloc(sizeof(int));
int *q = (int*)reallocf(p, sizeof(int));
if (p == q) {
*q = 2;
printf("%d %d\n", 2, 2);
}
return 0;
}
Note how it removed the write to p and removed the read of the pointer value.Here what it's done is assumed that because p == q, that means they alias, and therefore the write to p will be overwritten by the write to q, and that it doesn't have to read the value again to know what will be printed.
So the optimization here seems to be proceeding under the assumption that realloc() does not necessarily invalidate the pointer. And it behaves the same way with reallocf() as well.
cmp rbx, rax
jne .LBB0_2
mov dword ptr [rax], 2
mov dword ptr [rbx], 1
mov edi, .L.str
mov esi, 1
mov edx, 2
xor eax, eax
call printf
.LBB0_2:This unrelated string never should have been printed to the pipe in question in the first place (!), and also didn't even exist at the point where it got printed out - being calculated a few lines down (!!).
The issue went away when I fixed a seemingly unrelated bug (that didn't look like it involved undefined behavior at all), but it all still gives me nightmares to this day D:
The C is dark and full of terrors.
However, I'm having a bit of an issue understanding what the compiler is doing here, at the beginning of the article.
If someone can explain, it'd be appreciated.
FTA:
A post-classical compiler, on the other hand, might perform the following analysis:
The first four times through the loop, the function might return true.
When i is 4, the code performs undefined behavior. Since undefined behavior lets me do anything I want, I can totally ignore that case and proceed on the assumption that i is never 4. (If the assumption is violated, then something unpredictable happens, but that's okay, because undefined behavior grants me permission to be unpredictable.)
The case where i is 5 never occurs, because in order to get there, I first have to get through the case where i is 4, which I have already assumed cannot happen.
Therefore, all legal code paths return true.
As a result, a post-classical compiler can optimize the function to bool exists_in_table(int v)
{
return true;
}
I see might return true. Ignored. And never happens.I think the idea is that since i=4 is ignored, and the loop goes while i <= 4, you can never reach the return false statement? That's my understanding, I'm just not confident on it.
Basically, since i=4 causes undefined behaviour, the compiler assumes that it can't possibly happen[1] and the only way that it can't happen is if one of the prior (i < 4) checks were true.
Therefore all legal code paths (ie all code paths that don't result in undefined behaviour) return true.
So it optimises it to simply return true. Because otherwise i=4 would have happened, but that's undefined, so impossible[1]
[1] but if it does happen, that's ok, because that would be undefined behaviour, which allows the compiler to do whatever it feels like anyway
And, i=4 is not ignored by the compiler, it is declared as undefined behaviour. What the compiler MAY do is to always return true.
In this cause, it will always return true, either because it reached the 5th value ([4]), or because there was a match in the first 4 values.
As the result will always be true, the compiler may then simply return true and skip the loop.
In this particular case, the 'contract' and the actual code imply that "The only allowed values of 'int v' are those that actually are found in the table"; for those values the function correctly returns true; and for all the possible 'illegal' arguments any and every possible behavior would be correct.
Articles like this and the three-part series about undefined behavior on the LLVM blog [0] ought to be required reading for anyone who still has the impression that C is "portable assembler".
[0] http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
The compiler should, again in my opinion, in the presence of undefined behavior simply spit out an error and say "Behavior here is undefined, fix it." that any compiler could recognize some undefined behavior in the way the code was written, and exploit that as an "optimization" boggles my mind.
So one could conclude that the compiler has to prove termination of ring_bell() before performing the optimization discussed, which is impossible for just any external function.
Furthermore, when undefined behaviour is invoked anywhere within a program, the whole program is undefined.
It's assuming undefined means the code can never occur (so it removed that code), but aren't most programmers assuming the code can occur but something weird will be done?
The problem is that programmers probably assume that many things which are undefined are implementation defined.
An example of something that is implementation defined are struct layouts: the standard (I think) does say that the order must be the same as defined in the struct definition, but it allows the implementation to put as much space as it wants inbetween those values (for optimal architecture alignment.) Things that are implementation defined are typically going to be things which must happen, but if the standard defined exactly how, it would unnecessarily constrain the implementation (such as being able to perform optimizations).
So if you did the following:
int data[1];
int foo = data[1];
printf("Bar");
foo would be undefined, but you know that "Bar" would be printed regardless.My question is: are there any legitimate optimizations that would be prevented by this?
It's also worth noting that your trivial example would result in basically everything be removed, but most non-trivial examples don't do that. The most common 'optimization' from undefined-behaviour is that the compiler doesn't need to check for those conditions and can let whatever will happen happen, and that only works if it's defined in a program-wide anything-goes sense. If it's defined on a local sense, then if say 'data' was passed-in as a parameter instead of declared, the compiler would have to insert a NULL check to make sure no undefined-behaviour happens and the program doesn't crash (So that "Bar" prints). By defining undefined-behaviour like it is, there's no requirement for the compiler to do a NULL check, it can instead just assume the programmer will never let it happen and produce code with that in mind. Same thing with integer overflow and similar cases (Though things get a bit hairier there).
if(x){
foo();
}else{
undefined();
}
which can be optimized to foo()The compiler often can't verify it (halting problem and friends), but it allows for nice optimizations by assuming that the code as written is actually correct, and the check was skipped intentionally.
$ g++ -O2 -Wall -Werror -o asdf asdf.cpp
asdf.cpp: In function ‘int main(int, char**)’:
asdf.cpp:9:29: error: iteration 3u invokes undefined behavior [-Werror=aggressive-loop-optimizations]
std::cout << (i*1000000000) << std::endl;
^
asdf.cpp:7:2: note: containing loop
for (int i = 0; i < 4; ++i)
^
cc1plus: all warnings being treated as errors
Clang compiles it but doesn't produce an endless loop.You can get defined behaviour by casting to unsigned:
std::cout << (int)(((unsigned) i) * 1000000000U) << std::endl;
Which seems like a good rule of thumb: when working on x86 and x64 and doing things with numbers that you think might overflow, do it with unsigned and cast back to what you need. if (b)
a = *b;
else
a = 3;
Since dereferencing 0 is undefined, the compiler can assume that a = 3 never needs to be executed??But b may legitimately be 0 and then the second branch SHOULD be entered
How does this fit with what the author said? The compiler cant just go back and assume b is never zero just because it's being dereferenced, since the dereference is guarded.
That's why his last part doesnt make sense -- that even f you try to prevent a bad dereference, undefined behavior is triggered.
To invoke undefined behavior and strange optimizations, you'd need to rearrange the code a bit:
a = *b;
if (!b)
a = 3;
Here, the compiler can omit the if statement and its contents entirely, because b cannot be NULL, because the first line would invoke undefined behavior if it were.A check for NULL before you dereference is always safe. It's when you do it the other way around that the compiler can start doing strange things.