Thursday, 9 May 2019

assembly - What does `rep ret` mean?



I was testing some code on Visual Studio 2008 and noticed security_cookie. I can understand the point of it, but I don't understand what the purpose of this instruction is.



    rep ret /* REP to avoid AMD branch prediction penalty */


Of course I can understand the comment :) but what is this prefix exaclty doing in context with the ret and what happens if ecx is != 0? Apparently the loop count from ecx is ignored when I debug it, which is to be expected.




The code where I found this was here (injected by the compiler for security):



void __declspec(naked) __fastcall __security_check_cookie(UINT_PTR cookie)
{
/* x86 version written in asm to preserve all regs */
__asm {
cmp ecx, __security_cookie
jne failure
rep ret /* REP to avoid AMD branch prediction penalty */
failure:

jmp __report_gsfailure
}
}

Answer



There's a whole blog named after this instruction. And the first post describes the reason behind it: http://repzret.org/p/repzret/



Basically, there was an issue in the AMD's branch predictor when a single-byte ret immediately followed a conditional jump as in the code you quoted (and a few other situations), and the workaround was to add the rep prefix, which is ignored by CPU but fixes the predictor penalty.


No comments:

Post a Comment

php - file_get_contents shows unexpected output while reading a file

I want to output an inline jpg image as a base64 encoded string, however when I do this : $contents = file_get_contents($filename); print ...