hasher Flagyard
This challenge is from FlagYard, and it looks really interesting. I’m having a lot of fun solving it.
Okay, let’s begin. First, I loaded the sample into PEStudio to take a look at its imports
I noticed that IsDebuggerPresent is present in the imports, which is a good sign that the challenge has some anti-debugging tricks. There also seem to be a few other anti-analysis techniques, so I’m curious to see how they are actually being used.
For now, let’s run the challenge and see what happens. I want to check the output first before diving deeper into the analysis.
The output says that the flag has the wrong length, so first we need to figure out the correct length of the flag.
Here, we can see an array containing 39 characters, so I guessed that the flag should also be 39 characters long.
I checked my guess, and it turned out to be correct because the program moved past the wrong length check and gave me a wrong flag message instead
To be honest, I didn’t understand much of the code at first, so I decided to use the debugger to get a better idea of how the program works and understand its logic step by step
At this point, I first tried using ScyllaHide to bypass the anti-debugging techniques, but it didn’t work as expected. The program would call sub_140001080 and then stop.
So, I decided to patch the anti-debugging check by replacing the instruction with a NOP (No Operation). This allowed me to bypass the check and continue debugging the progr
Now I reached another interesting part. The program checks the value in RAX and compares it with 0x27, which is 39 in decimal—the length of our input.
Get Chadi’s stories in your inbox
Join Medium for free to get updates from this writer.
So, to get past this check while debugging, I patched the conditional jump by replacing the Je instruction with a jne, allowing the program to continue
I also installed a plugin that makes the disassembly easier to navigate, similar to IDA Pro, especially when dealing with jump tables. It makes following the program’s control flow much easier
After that, the program takes the value of AL and puts it on the stack. Since AL contains the first character of our input, I decided to use 0x46, which represents F, so we can see exactly what the program is doing with this character.
here rax is zero
At this point, I put F into RAX and continued through the operations. The sequence of operations was quite long, and even though I wrote them all down on paper, I realized that doing everything manually wasn’t really practical.
That’s what I did for this part, but if you found a better or more efficient way to approach it, feel free to leave a comment!
To save some time, I noticed that the whole operation is basically a repeated pattern: shift left → XOR → shift right → add.
This pattern is repeated 8 times, so doing it manually would be pretty annoying. Instead, I wrote a small script to reverse these operations and recover the original character value
It was a really good challenge, and I genuinely enjoyed solving it. If you have any questions or notice something I could have done better, feel free to leave a comment, and I’ll try my best to help.
Thanks for reading, and see you in the next challenge
