//nefariousplan

CVE-2026-88771: NetScaler's Audit Log Is the Shell. The Logger Runs as Root.

patterns

cve

proof of concept

A failed login to NetScaler's management interface writes the attacker's username, verbatim, to /var/log/ns.log as a CMD_EXECUTED audit record. Twenty-four hours later, a root-owned Perl script named ns_monuploadd_err.pl greps that log for packet-engine crash signatures. The username was the crash signature. The characters that survived sed are the ones the shell interprets.

The record that convicted its own reader

A NetScaler ADC appliance keeps an audit log at /var/log/ns.log. Every failed login writes a line in a documented format: timestamp, host, facility, CMD_EXECUTED:, then the submitted username, then the (redacted) command, then a status. The record is the whole point of audit logging. If an attacker tries a hundred thousand usernames against the management portal, a hundred thousand audit lines appear. A SOC analyst scrolling through 401s was supposed to be the reader of that log.

On NetScaler ADC 14.1 before 14.1-73.37 and 13.1 before 13.1-64.23, the log had a second reader. The second reader was a root-owned Perl script called /netscaler/ns_monuploadd_err.pl. It ran as a child of admautoregd, the NetScaler Console registration daemon, which itself was started from nsstartpe.sh with no privilege drop. The script was invoked with the flag -WR, meaning "warm restart diagnostics." Its job was to grep the log for packet-engine crash signatures, pick the last matching line, parse out a core-dump filename, and find the dump on disk so a telemetry worker could upload it.

The username field in a failed CMD_EXECUTED: record and the format of a packet-engine crash line were, from the parser's perspective, the same shape. The username was supplied by an unauthenticated HTTP client. The parser interpolated the derived "filename" into a backtick find without quoting. The capture of a failed login was the input to a root shell, on a 24-hour timer.

That is CVE-2026-88771. Citrix patched it on 2026-08-26. CISA added it to the Known Exploited Vulnerabilities catalog with a federal remediation deadline of 2026-09-30. By the time the KEV entry landed, watchTowr had already published a working exploit, Sliver C2 traffic had been observed in the wild to 176.94.85.67:80, and defenders were being told to hunt for log lines containing pitboss followed by ${IFS} or a shell metacharacter. The patch Citrix shipped does two things. It tightens the parser so that unexpected input can no longer survive sed. It also removes the caller. The call site that ran the parser as root at daemon startup is gone from 14.1-73.37's probe.py. Two layers of walkback for one input validation bug is the shape of a vendor that no longer trusted the primitive after they fixed it.

What a failed login looks like after it is logged

The authentication endpoint on a NetScaler Gateway is /nf/auth/doAuthentication.do. It accepts a login= form parameter and a passwd= form parameter. The management NITRO interface accepts the same credentials as JSON at /nitro/v1/config/login. Both paths produce the same audit record when the submitted username fails to authenticate. The format the audit formatter prints is this:

User <username> - ADM_User <admuser> - Remote_ip <ip> - Command "login <username> \"********\"" - Status "ERROR"

The password is redacted. The username is not. The username is written three times into the record, verbatim, with no character allowlist and a length cap of 0x80 bytes enforced on the request-parsing side of nsppe. There is no HTML escaping, no shell escaping, no normalisation. The audit log is a dump of raw request bytes framed by User and - ADM_User.

The watchTowr public exploit builds its username field as follows:

login_payload = f"pitboss PPE unexpectedly died NSPPE;{args.command};# X"
encoded_login = urllib.parse.quote(login_payload, safe='')
url = args.target.rstrip('/') + "/nf/auth/doAuthentication.do"
body = (f"login={encoded_login}&passwd=x&savecredentials=false"
        f"&nsg-x1-logon-button=Log+On")

That POST returns a 401. The user did not exist. The audit log now contains, on one line, the attacker's chosen pitboss PPE unexpectedly died NSPPE;id>/var/tmp/watchTowr;# X as the username field. The attacker has written one audit line and is done. Nothing on the appliance has executed yet. The attacker's job is finished. This is Content Is Command in its purest form: the content of a web form field is the input to a shell that runs 24 hours later as root.

The second reader is a Perl backtick

The vulnerable script at /netscaler/ns_monuploadd_err.pl version 14.1-72.61 contains this block:

my $WR_PPE_COREFILE_NAME = `grep -E -i "pitboss.*PPE.*missed too many heartbeats|pitboss.*PPE.*unexpectedly died" @WR_FILES |\
tail -1 | sed -e 's!.*NSPPE!NSPPE!g' -e 's!(!!g' -e 's!)!!g' | awk '{ print \$1"-"\$2 }'`;
my $WR_PPE_CORE = `find $OFF_DIR/var/core -name ${WR_PPE_COREFILE_NAME}* -print | tail -1`;

The first backtick runs a shell pipeline against the log files. The grep expression is deliberately loose. The sed chain replaces everything up to and including the last NSPPE with the literal string NSPPE, strips any parentheses, and the awk prints the first two whitespace-delimited tokens joined by a hyphen. The result is captured into $WR_PPE_COREFILE_NAME and then interpolated, unquoted, into a second backtick that runs find.

Trace the watchTowr username through that pipeline:

  1. The audit line in ns.log contains, among other things, pitboss PPE unexpectedly died NSPPE;id>/var/tmp/watchTowr;# X. The grep matches on pitboss.*PPE.*unexpectedly died.
  2. tail -1 keeps the latest matching line. In an active appliance that is almost always the attacker's line, because real pitboss warm-restart messages are rare and the attacker just wrote one at request time.
  3. sed 's!.*NSPPE!NSPPE!g' strips the line's prefix. What remains is NSPPE;id>/var/tmp/watchTowr;# X.
  4. The two parentheses-stripping sed stages do nothing to this input, because the attacker chose a payload with no ( or ). That constraint is the reason exploit authors publish a "sink-safe payload rules" list rather than an arbitrary command.
  5. awk '{ print $1"-"$2 }' prints the first two whitespace-delimited fields joined by -. Field one is NSPPE;id>/var/tmp/watchTowr;#. Field two is X. The result is NSPPE;id>/var/tmp/watchTowr;#-X.
  6. That string lands, unquoted, inside the second backtick. Perl expands it in place. The shell receives: find /var/core -name NSPPE;id>/var/tmp/watchTowr;#-X* -print | tail -1.
  7. The shell splits on ;. The first command, find /var/core -name NSPPE, runs and fails harmlessly. The second command, id>/var/tmp/watchTowr, runs as root. The third component, beginning with #, is a comment that consumes the trailing -X* -print | tail -1.

The attacker's command executes in a context where real pitboss logging had already carved out exactly the characters needed to express arbitrary shell. There is no quoting at the Perl layer, no quoting at the sed layer, no quoting at the awk layer. The pipeline was designed as if the input were a trusted packet-engine kernel message. The input is a form field from an unauthenticated POST.

Why it takes twenty-four hours and why that is the vendor's problem

ns_monuploadd_err.pl -WR is not invoked synchronously by the login handler. It is invoked by admautoregd's telemetry probe. In mastools/scripts/admautoreg/probe.py at version 14.1-72.61, the probe schedules itself for the first time at daemon_start + 1800 seconds, then every 86400 seconds thereafter. The constant that names the script lives in the same package at admautoreg/constant.py:146:

CHECK_PE_FAILURE_CMD = '/netscaler/ns_monuploadd_err.pl -WR'

Twenty-four hours of latency is the practical delivery window, and it is why the CVSS v4 vector includes AT:P, Attack Requirements: Present. The exploit itself is reliable. The delivery is not interactive. The attacker POSTs a payload, closes the connection, and waits for the appliance's own privileged scheduler to run the shell for them. A reboot before the scheduled cycle does not save the defender. The script runs at daemon start as well. A SOC analyst who reviews the day's CMD_EXECUTED audit entries and finds a weird pitboss username has already lost the box, because the box is going to execute the username on its own clock.

watchTowr's PoC notes "it might take up to 24 hours for the command to get picked up," and ships with an OOB callback mode so the operator can confirm execution later without authenticating. The craigsblackie trigger PoC enumerates the sink's character constraints explicitly: no (, no ), no raw space (use ${IFS}), no ?, no &. The constraints are the shape of the sed and awk stages. Any command that fits them survives.

The patch is two walkbacks, not one

The 14.1-73.37 version of ns_monuploadd_err.pl replaces the backtick pipeline with Perl open in list form and a captured regex:

my $re = qr/pitboss.*(NSPPE-\d{2})\s*\((\d+)\).*(missed too many heartbeats|unexpectedly died)/;
open my $find, '-|', 'grep', '-E', '-i', $pattern, @WR_FILES
    or die "open failed: $!";
while (<$find>) {
    if (/$re/) {
        $derived = "$1-$2";
    }
}

Two capture groups. The first is anchored to the literal string NSPPE- followed by exactly two digits. The second is anchored to a parenthesised numeric packet-engine ID. Anything else in the line is discarded. The attacker's semicolon and payload no longer survive, because they are not matched by any capture group. The regex is restrictive by construction. The subsequent file access uses -| list-form open, which bypasses the shell entirely.

That alone fixes the vulnerability. It is not the whole patch.

Reading admautoreg/probe.py and admautoreg/constant.py across 72.61 and 73.37, the following constants and functions are present at 72.61 and absent at 73.37:

  • CHECK_DISK_FAILURE_CMD
  • CHECK_PE_FAILURE_CMD
  • _check_disk_pe_failure()
  • get_disk_failure_info()
  • get_pe_failure_info()

The caller is gone. The telemetry probe no longer invokes ns_monuploadd_err.pl -WR at all. The script survives on disk, and whatever manual invocation path exists for a running administrator still works, but the automatic root-at-daemon-start invocation that gave the bug its 24-hour timer no longer exists.

Two layers of walkback for one input validation bug is Citrix's own assessment of the design. The parser was unsafe. The caller was also unsafe for calling the parser against attacker-reachable input. The fix removes both.

The pattern: the detector was the attack surface

This is the Zabbix shape. In 2024, CVE-2024-22120 made Zabbix's audit log the primitive: a privileged job scheduler read audit records with an unsafe query, and attacker-controllable input to the audit record became SQL that ran at the auditor's privilege. nefariousplan tagged that case under the-detector-is-the-target, with the editorial line The audit log captured the attack faithfully. The capture is the attack.

NetScaler's version is louder. Zabbix's logger ran as the database user. NetScaler's logger ran as root. Zabbix's reader was a scheduled query. NetScaler's reader was a shell. Zabbix's attacker had to be authenticated. NetScaler's attacker needed only an HTTPS reachable to the management interface and a username field that passes the input filter at the request layer, which rejects nothing more selective than "fewer than 128 bytes."

The audit log is, in both cases, a store of attacker-authored strings that the vendor's own code reads back privileged. The engineering mistake in both cases is treating the log as a diagnostic artefact rather than an attack surface. The log is a database. The attacker is a writer. The reader is a program that trusts the writer.

A hunt rule follows directly. Any pitboss.*PPE.*(missed too many heartbeats|unexpectedly died) line in /var/log/ns.log that is followed on the same line by a shell metacharacter in [;|\$], or by the literal string ${IFS}, is an exploit attempt. The SwiftSecur hunt script encodes exactly that regex. On a compromised appliance, the hunt also looks for the write-side evidence: /var/tmp/wtw888, /var/tmp/cve88771, /var/walk, a dotfile named .local_journal, and backdoor local accounts named gw_health`.

The disclosure order

Citrix's patch landed on 2026-08-26. The CVE was reserved and published alongside the fix. watchTowr published a working exploit before the KEV deadline of 2026-09-30. Multiple vendors reported in-wild Sliver C2 traffic (176.94.85.67:80), post-exploitation OOB callbacks to *.ctrxsrv.com, *.dnshook.site, and *.oast.fun, and 47 distinct source IPs probing failed-login endpoints with pitboss usernames.

The order was: patch, in-wild, KEV. Disclosure After Exploitation is the ordinary KEV sequence; the catalog documents an exploitation window that was already open when the vendor advisory went out. For NetScaler defenders this is the fourth time in two years that a management-plane unauth RCE has followed the same shape: a privileged daemon reaches an attacker-reachable input through a parser that treated external text as internal protocol. The vendor's recurring class is a Design Debt Driver. Each CVE is an exhibit in it, and the exhibits now cover authentication, SSL VPN session parsing, management-plane deserialisation, and audit-log reparse.

The appliance in question is a load balancer at the edge of a corporate network. Its purpose is to see all authenticated traffic to the services behind it. The box that reads the audit log is the box that terminates TLS for every internal application. A root shell on the box that terminates TLS for every internal application is, by construction, a root shell on the audit itself. The capture has become the attack, and the capture has become the control plane.

Appendix: the signatures that matter

Vulnerable /netscaler/ns_monuploadd_err.pl on 66.59 and 72.61: SHA-256 fb7f574a4c185fa8e520c47280939ce22899243a0083ee7120b7300c43baca29.

Patched /netscaler/ns_monuploadd_err.pl on 73.37: SHA-256 02b24a9923a6cee1fbee48137b7f81b7e0f5169246728b7368fc3a066c43a708.

The two hashes are identical bytes between 66.59 and 72.61. The vulnerable primitive did not change for the full 14.1 release train until it was removed. A grep rule on /var/log/ns.log for pitboss adjacent to ${IFS} or [;|\$]` detects the exploit on both pre-patch versions. The absence of the caller in 73.37 is the vendor's own documentation that the primitive was unsafe at the architectural level, not only at the string-handling level.

The capture is the attack. On NetScaler, the capture runs as root.