Table of Contents:
- Introduction (1/12)
- The Easy Way (2/12)
- Basics (3/12)
- Debugging (4/12)
- Hooking Winsock (5/12)
- Egress – Walking the Call Stack (6/12)
- Egress – Reversing the Request Encrypt Function (7/12)
- Egress – Monitoring (8/12)
- Ingress – Walking the Call Stack (9/12)
- Ingress – Reversing the Response Decrypt Function (10/12)
- Ingress – Monitoring (11/12)
- Conclusion (12/12)
- Source code
Having identified some promising leads in the previous post, it is time to start the investigation. To begin with, the Xbox Live API functions HCHttpCallRequestSetUrl, HCHttpCallRequestSetHeader, HCHttpCallRequestSetRequestBodyBytes, and similar all look very promising. If they are indeed used for the client-server communication that we are interested in, then it should be a simple matter of hooking into them and getting a hold of the HTTP content.
We can begin by attaching x64dbg to the Age of Empires IV process (RelicCardinal.exe), and finding what functions reference these strings. Lets start with HCHttpCallRequestSetUrl. In this case, there is only one location:
00007FF64CC88673 | 48:C7C3 FFFFFFFF | mov rbx,FFFFFFFFFFFFFFFF |
00007FF64CC8867A | 4C:8BC3 | mov r8,rbx |
00007FF64CC8867D | 0F1F00 | nop dword ptr ds:[rax],eax |
00007FF64CC88680 | 49:FFC0 | inc r8 |
00007FF64CC88683 | 42:803C07 00 | cmp byte ptr ds:[rdi+r8],0 |
00007FF64CC88688 | 75 F6 | jne reliccardinal.7FF64CC88680 |
00007FF64CC8868A | 49:8D4E 08 | lea rcx,qword ptr ds:[r14+8] |
00007FF64CC8868E | 48:8BD7 | mov rdx,rdi |
00007FF64CC88691 | E8 9AEBFFFF | call reliccardinal.7FF64CC87230 |
00007FF64CC88696 | 90 | nop |
00007FF64CC88697 | 4C:8BC3 | mov r8,rbx |
00007FF64CC8869A | 66:0F1F4400 00 | nop word ptr ds:[rax+rax],ax |
00007FF64CC886A0 | 49:FFC0 | inc r8 |
00007FF64CC886A3 | 42:803C06 00 | cmp byte ptr ds:[rsi+r8],0 |
00007FF64CC886A8 | 75 F6 | jne reliccardinal.7FF64CC886A0 |
00007FF64CC886AA | 49:8D4E 28 | lea rcx,qword ptr ds:[r14+28] |
00007FF64CC886AE | 48:8BD6 | mov rdx,rsi |
00007FF64CC886B1 | E8 7AEBFFFF | call reliccardinal.7FF64CC87230 |
00007FF64CC886B6 | 90 | nop |
00007FF64CC886B7 | 41:80BE 20010000 00 | cmp byte ptr ds:[r14+120],0 |
00007FF64CC886BF | 74 2A | je reliccardinal.7FF64CC886EB |
00007FF64CC886C1 | 48:897424 28 | mov qword ptr ss:[rsp+28],rsi | [rsp+28]:"
00007FF64CC886C6 | 48:897C24 20 | mov qword ptr ss:[rsp+20],rdi |
00007FF64CC886CB | 4D:8B8E 18010000 | mov r9,qword ptr ds:[r14+118] |
00007FF64CC886D2 | 4C:8D05 0FAD3103 | lea r8,qword ptr ds:[7FF64FFA33E8] | 00007FF64FFA33E8:"HCHttpCallRequestSetUrl [ID %llu]: method=%s url=%s"
00007FF64CC886D9 | BA 04000000 | mov edx,4 |
00007FF64CC886DE | 48:8D0D DB51EF04 | lea rcx,qword ptr ds:[7FF651B7D8C0] | 00007FF651B7D8C0:&"HTTPCLIENT"
00007FF64CC886E5 | E8 46ADFFFF | call reliccardinal.7FF64CC83430 |Lets begin by taking a look at the structure of these instructions. We can see that there are two blocks [00007FF64CC88673 to 00007FF64CC88691] and [00007FF64CC88697 to 00007FF64CC886B1], which perform very similar functionality: they load some parameters, and call reliccardinal.7FF64CC87230. To analyze this assembly listing, we will start bottom-up and look at the block beginning at 00007FF64CC88697. Why do it this way? Because it is closer to the instructions that print out information related to HCHttpCallRequestSetUrl.
Here, at 00007FF64CC886B1, we call reliccardinal.7FF64CC87230, and then check if [R12+0x120] is equal to 0. If it is not, we skip calling next set of instructions; otherwise we continue executing and call reliccardinal.7FF64CC83430. The format string provides a lot of information about the arguments of this function. We can deduce that [R14+0x118] holds the request ID, [RSP+0x20] holds the method, and [RSP+0x28] holds the URL. Following the x64 calling convention, and taking hints from the disassembly, we can deduce that the function call looks similar to
reliccardinal_7FF64CC83430((void *)0x7FF651B7D8C0, 4, "HCHttpCallRequestSetUrl [ID %llu]: method=%s url=%s",
id, method, url);This is a good start. We can now work backwards to learn more. If we look at the call to reliccardinal.7FF64CC87230 that occurs at 00007FF64CC886B1, we see that it takes three arguments: [R14+0x28], and the RSI and R8 registers. The RSI register was loaded into [RSP+28], the last argument of reliccardinal.7FF64CC83430, which we have determined is the request URL. The R8 register appears to be calculated based off of RSI; it is incremented until it reaches a place where [RSI+R8] is 0. Since RSI is a string containing the URL, then R8 must be calculating where the null terminator is, effectively calculating the length of the URL. The value of [R14+0x28] is still undetermined, but if we look higher in the disassembly, towards the beginning of the function, we see these instructions:
00007FF64CC885D9 | 49:8BF0 | mov rsi,r8 |
00007FF64CC885DC | 48:8BFA | mov rdi,rdx |
00007FF64CC885DF | 4C:8BF1 | mov r14,rcx | The R14 register gets assigned here to RCX, the first argument of this function we are reversing, and doesn’t get re-assigned at any other point in the function. Different offsets into R14 get referenced and passed as arguments to other functions, meaning that it is some kind of structure. We can piece together what it is by looking at the HCHttpCallRequestSetUrl documentation and discover that it is the HCCallHandle opaque pointer. We also see that RDI and RSI get assigned here as well to the second and third arguments respectively. Since we have the function definition from the documentation we can conclude that RDI holds the method and RSI holds the URL. This is consistent with what we found from looking deeper into the function disassembly.
This gives us enough information to see what this block is doing: it is taking the input URL and copying it to an internal buffer inside the HCCallHandle structure. We can apply this logic for the other, nearly identical, block beginning at 00007FF64CC88673 and see that it is doing the same thing for the HTTP method parameter.
Putting all of this together allows us to begin reconstructing the original code of this function. Taking what we know, we can translate the above assembly listing into something that looks like this:
...
WriteToBuffer(callHandle->MethodBuffer, method, strlen(method));
WriteToBuffer(callHandle->UrlBuffer, url, strlen(url));
if (callHandle->ShowDebugOutput) {
DebugWrite((void *)0x7FF651B7D8C0, 4, "HCHttpCallRequestSetUrl [ID %llu]: method=%s,
url=%s", callHandle->Id, callHandle->MethodBuffer, callHandle->UrlBuffer);
}
...
With the debugger still attached, we can set breakpoints on where the two writes happen. If the breakpoints get hit, we should be able to get the HTTP method and URL for the outgoing request. So lets do that: set the breakpoints, tab back into the game, and begin performing some actions. If you do that, you will unfortunately find out that the breakpoints never get hit. The game does not appear to be using this function. It is the same story for the other HCHttpCallX functions; the request logic must be happening somewhere else.
Having spent some time on a fruitless search, we can take another approach. Instead of trying to investigate library internals, we can look one level higher at the Windows API. The functions that we are interested in are the send and recv functions provided by the Windows sockets API. Barring some very atypical implementation, all networking functionality will go through these two functions; send will be called to send data out to a socket, and recv will be used to read data from a socket. Lets verify this by setting a breakpoint on send.
Set the breakpoint and perform some activity that would require getting data from the server, i.e. refresh the available game lobbies. If the breakpoint has been successfully set, it should have been hit.
This is a great sign: we have some point at which to begin really reverse engineering and stepping backwards from. We know that the HTTP data must have been constructed at some point prior to calling send. We can take a quick detour and come up with a plan of action. To start with, we can hook the send and recv APIs and dump out the outgoing and incoming buffers. To do this, we will be using Microsoft’s Detours library. We will write this in a DLL that will be injected into the Age of Empires IV process. From within this DLL, we will hook send and recv and redirect them to our hook functions. From these hook functions we will dump out the buffers to a console. To start with, we can write the DllMain function to do this:
__declspec(dllexport) BOOL WINAPI DllMain(HINSTANCE hModule, DWORD dwReason, LPVOID reserved)
{
static HookEngine hookEngine{};
if (dwReason == DLL_PROCESS_ATTACH) {
DisableThreadLibraryCalls(hModule);
if (AllocConsole()) {
(void)freopen("CONOUT$", "w", stdout);
(void)freopen("CONOUT$", "w", stderr);
SetConsoleTitle(L"Console");
SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE);
std::cerr << "DLL Loaded" << std::endl;
}
(void)hookEngine.Hook("Ws2_32.dll", "send", sendHook);
(void)hookEngine.Hook("Ws2_32.dll", "recv", recvHook);
}
if (dwReason == DLL_PROCESS_DETACH) {
(void)hookEngine.Unhook("Ws2_32.dll", "send");
(void)hookEngine.Unhook("Ws2_32.dll", "recv");
}
return TRUE;
}Upon attaching to the process, we will allocate a console that we will write the data buffers to. We then hook the send and recv functions in Ws2_32.dll. The HookEngine class is just a wrapper around the Detours API. The Hook function is implemented as follows:
bool HookEngine::Hook(std::string_view moduleName, std::string_view functionName, HookFncPtr hookAddress)
{
if (IsHooked(moduleName, functionName)) {
std::cerr << std::format("{}:{} is already hooked.", moduleName, functionName)
<< std::endl;
return false;
}
auto functionAddress{ GetFunctionAddress(moduleName, functionName) };
if (functionAddress == nullptr) {
std::cerr << std::format("Hook installation failed. Address for {}:{} is nullptr.", moduleName, functionName)
<< std::endl;
return false;
}
auto result{ Hook(functionAddress, hookAddress) };
if (!result) {
return false;
}
m_hookedFunctions[std::string{ moduleName }].push_back(std::make_pair(std::string{ functionName }, functionAddress));
std::cerr << std::format("Hook installed on {}:{} successfully.", moduleName, functionName)
<< std::endl;
return true;
}
bool HookEngine::Hook(FncPtr originalAddress, HookFncPtr hookAddress)
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)originalAddress, (PVOID)hookAddress);
auto result{ DetourTransactionCommit() };
if (result != NO_ERROR) {
std::cerr << std::format("Hook transaction failed with code {}.", result) << std::endl;
return false;
}
m_hookToOriginal[hookAddress] = originalAddress;
return true;
}For more details on the implementation, you can view the source listing in the Github link provided in the table of contents at the top of this post.
We define the sendHook and recvHook functions, which will be called when the target process calls send and recv. These functions do nothing except print out the incoming and outgoing buffers in a nice format.
template <typename ReturnType, typename FunctionType>
ReturnType PassthroughHook(void *caller, SOCKET socket, char *buffer, int length, int flags)
{
if (!IsIgnoredPacket(buffer, length)) {
auto output{ MakePrintableAscii(buffer, length) };
auto [ipAddress, port] { GetPeerInfo(socket) };
for (const auto& line : output) {
std::cerr << std::format("[{}:{}] - Data: {}", ipAddress, port, line)
<< std::endl;
}
}
auto original{ (FunctionType)HookEngine::GetOriginalAddressFromHook(caller) };
ReturnType result{};
if (original != nullptr) {
result = original(socket, buffer, length, flags);
}
return result;
}
int WSAAPI sendHook(SOCKET socket, const char* buffer, int length, int flags)
{
return PassthroughHook<int, sendFnc>(sendHook, socket, const_cast<char*>(buffer), length, flags);
}
int WSAAPI recvHook(SOCKET socket, char* buffer, int length, int flags)
{
return PassthroughHook<int, recvFnc>(recvHook, socket, buffer, length, flags);
}We also define an IsIgnoredPacket function that prevents printing certain data buffers. This is to prevent our screen from being flooded with calls that happen constantly, i.e. heartbeats and other calls that we are not interested in. Building this DLL and injecting it into the process shows that the hooks are being called. We can see data coming in and going out from the process. However, when looking at it, we can’t really see anything useful.
There is some occasional information that is visible in plaintext, but it is hard to get the context of the data and what it matches up to.
This is a good start, but we have cast too wide of a net. By hooking send and recv, we are monitoring everything coming in and going out over the network. If the goal is to get at the REST APIs, we need to go further down in the code and isolate the logic responsible for them. That will be the topic of the next posts.