The challenge gave us a file call rop.iseq. By checking the file header, I found that it was a binary format of Ruby's InstructionSequence.
By googling the InstructionSequence, I found that there are some new features were added into the ruby version 2.3, for example the load_from_binary method. We can actually use these methods to load the instruction sequence from a binary file, and disassemble the instruction to a human readable format.
If we execute the line d.eval, it will run the instruction sequence:
Looks like the program will read our input and do some checking, then output the checking result.
Anyway let's dump the disassemble result and start reversing. Here's the whole disassemble result.
Google is our friend. I found a useful reference for introducing basic ruby instruction sequence reversing.
For example for the following iseq:
trace 1 means "A new line of Ruby code has been encountered". Then by reading the following lines, we know that the line of the code was probably require "digest".
And so we can try to reverse the whole iseq by following the similar pattern. First we found the code that read the user input:
So local_OP__WC__0 3 will be our input. Now for the first check:
We can see that the valid key format must be something like "X-X-X-X-X". Here I also found a sequence of iseq which help us infer the precise key format:
So now we know that the key format is "XXXX-XXXX-XXXX-XXXX-XXXX", while "X" is in the range of [0-9A-F]. Time to recover the valid key.
The checking of the first part of the key was pretty simple:
So key is hex(31337) = 7A69
The checking of the second part of the key is even more simple:
So key = "FACE".reverse = ECAF.
To verify if key and key were the right value, we can actually use the following command to trace the ruby code: ruby -r tracer de.rb. If the key was correct, it would perform more checking, which means it will execute more line of code, so we can know if a part of the key was right or wrong by observing the trace of the ruby tracer ( kind of a side-channel analysis. )
Back to our recovering procedure. The checking of the key looks like this:
It will first call a method f, with argument (217, key.to_i(16), 314159), then check if its return value = 94449 ( with 28 as base, 48D5 is actually 94449 in base 10 )
method f was kind of complicated, so I will just post the pseudo code instead:
Since we know that key's format is 0000 ~ FFFF, we can just crack key by writing a simple crackme:
And so we got the value of key: 1BD2
Moving on to the next part (key):
At first I was confused at line 0216 ~ 0218. There's a :first for map, but the argc of map was actually 0. After doing some search on the internet, I found this post and found out that the check was actually doing:
So the value of key is 53*97 == 5141 ( base 10 )
At this point we know the valid key is 7A69-ECAF-1BD2-5141-XXXX. The checking of the last part of the key was also kind of complicated and I was kind of lazy to reverse the whole thing. So far we have the first four part of the key, and there's only one left ...... so why don't we use the old typical brute force attack to recover the last one ? ;)
And after about 20 minutes....
Looks like I should brute force the key from 0xffff down to 0 though :P
Anyway, the valid key is 7A69-ECAF-1BD2-5141-CA72, and so we got the flag !