The frontend bind serves a caller that does not exist
SchedulerServer.__init__, in archive/ktransformers/server/balance_serve/sched_rpc.py:
class SchedulerServer:
def __init__(self, settings, main_args):
self.sched = sched_ext.create_scheduler(settings)
self.context = zmq.Context()
self.frontend = self.context.socket(zmq.ROUTER)
print(f"sched zmq rpc server on port {main_args.sched_port}")
self.frontend.bind(f"tcp://*:{main_args.sched_port}")
# 创建内部的 DEALER 套接字，用于与工作线程通信
self.backend = self.context.socket(zmq.DEALER)
self.backend.bind("inproc://backend")
The backend DEALER is bound to inproc://backend, in-process only. The author marked it with a Chinese comment that reads "create internal DEALER socket for worker thread communication." The frontend ROUTER, bound to every TCP interface, has no such comment. The author had a category in mind for what counted as internal. The frontend was outside it.
Six classes later in the same file:
class SchedulerClient:
def __init__(self, sched_port):
address = f'tcp://localhost:{sched_port}'
self.address = address
self.context = zmq.Context()
self.socket = self.context.socket(zmq.REQ)
self.socket.connect(self.address)
print(f"Connected to server at {self.address}")
The client is hardcoded to localhost. There is no other client. Grepping SchedulerClient across the repo returns four call sites and they all live in server/backend/interfaces/balance_serve.py, in the same Python process tree as the server. The legitimate caller of the scheduler RPC is, in every code path the project ships, the webserver running next to the scheduler.
The only client of the scheduler RPC connects to localhost. The bind to every interface was for the user the codebase never had.
The deployment guide is the disclosure
doc/en/balance-serve.md, line 49:
docker run -it --gpus all --privileged --shm-size 64g --name ktrans \
--network=host -v /mnt:/mnt \
approachingai/ktransformers:v0.2.4-AVX512 /bin/bash
Five flags do work here. --gpus all, --privileged, --shm-size 64g, -v /mnt:/mnt, and --network=host. The first four hand the container the resources it needs. The fifth puts the unauthenticated ROUTER socket on every interface of the host. There is no example anywhere in the docs that uses -p 8080:8080 or any port-mapped alternative. --network=host is the documented default deployment.
The "internal-only" status of the scheduler RPC is held by neither the bind address, nor the deployment recipe, nor any check the worker thread runs. It is held by the operator's understanding of which Docker flag they pasted.
The default sched_port is 56441, written into archive/ktransformers/configs/config.yaml. The port is not random across deployments. It is the same number on every default install. Port discovery is not a meaningful obstacle.
The operator profile reinforces the exposure. KTransformers is built for self-hosted LLM inference on heterogeneous CPU+GPU hardware, and its README pitches "consumer hardware" deployments alongside multi-GPU clusters. People run it on shared lab boxes, on cloud GPU instances rented for the afternoon, on workstations behind VPNs that researchers think of as private. None of those environments treat --network=host on port 56441 as a hostile decision because the docs do not flag it as one.
Pickle runs on both ends
The worker, after the bind:
def worker_routine(self):
worker = self.context.socket(zmq.REP)
worker.connect("inproc://backend")
while True:
try:
message = worker.recv()
data = pickle.loads(message)
method = data.get('method')
params = data.get('params', {})
...
pickle.loads invokes __reduce__ on the deserialized objects during deserialization. The chocapikk PoC payload is the textbook shape:
class RCE:
def __reduce__(self):
return (os.system, ('id > /tmp/ktransformers_pwned',))
Send pickle.dumps(RCE()) to the ZMQ ROUTER on port 56441. The worker calls os.system('id > /tmp/ktransformers_pwned') with the privileges of the ktransformers process before reaching the line that asks data.get('method'). There are ten worker threads waiting for messages, started by start_rpc_service:
for _ in range(10):
threading.Thread(target=self.worker_routine, daemon=True).start()
Ten concurrent RCE handlers. The architecture of a service designed to be hit at scale.
The same pickle channel runs the other direction. SchedulerClient.send_request:
self.socket.send(pickle.dumps(request))
response = self.socket.recv()
response = pickle.loads(response)
The webserver pickle.loads whatever the scheduler returns. A scheduler that has been compromised through the network primitive can RCE the webserver in the same way. The pre-patch KV-cache rebuild does not even need pickle gadgetry on the response side:
def rebuild_inferece_context(self, response):
data = response.get('inference_context')
inference_context = sched_ext.InferenceContext()
inference_context.k_cache = [fn(*args) for fn, args in data['k_cache']]
inference_context.v_cache = [fn(*args) for fn, args in data['v_cache']]
The client iterates data['k_cache'] and calls each entry as fn(*args). Whatever functions and arguments the server says are tensor-rebuild instructions, the client invokes. The function-call primitive is part of the API contract.
The chocapikk PoC stops at server-side execution. It does not chain into the webserver, because there is no need: the worker process can already read whatever the user is inferencing, write to disk, and reach whatever the host's outbound network allows. A weaponized chain that wanted the webserver process specifically (to tamper with completions returned to API clients, for example) would use the second pickle vector, the response one, and never touch the network primitive a second time.
The exception handler catches what already executed
except Exception as e:
response = {'status': 'error', 'message': str(e)}
worker.send(pickle.dumps(response))
The try block wraps data = pickle.loads(message). The except runs after pickle.loads returns or raises. By the time Exception is bound to e, the __reduce__ callable in the attacker's payload has already been invoked. The exception handler is graceful degradation for malformed messages from the legitimate client. The author was thinking about their own bugs, not the attacker's. The catch logs an error after the attacker has run code.
The fix preserves the all-interfaces bind as a configurable option
PR #1944 makes three changes. The bind:
- print(f"sched zmq rpc server on port {main_args.sched_port}")
- self.frontend.bind(f"tcp://*:{main_args.sched_port}")
+ bind_addr = getattr(main_args, 'sched_bind', '127.0.0.1')
+ print(f"sched zmq rpc server on {bind_addr}:{main_args.sched_port}")
+ self.frontend.bind(f"tcp://{bind_addr}:{main_args.sched_port}")
tcp://*: is gone, replaced by a configurable sched_bind whose default is 127.0.0.1. The all-interfaces bind is not removed from the codebase. It is opt-in via main_args.sched_bind = '0.0.0.0'. Operators who set the new field back to the original behavior re-acquire CVE-2026-26210 with the framework's blessing.
The pickle handler is wrapped in a HMAC-signed multipart frame and a RestrictedUnpickler whose allowlist names the C++ scheduler extension types and a small set of stdlib classes:
_ALLOWED_MODULES = {
"builtins": {"dict", "list", "tuple", "set", "frozenset", "int", "float",
"str", "bytes", "bool", "NoneType", "complex", "range",
"slice", "type"},
"collections": {"OrderedDict", "defaultdict"},
"datetime": {"datetime", "timedelta", "date", "time"},
"ktransformers.server.balance_serve.settings": {"*"},
}
class RestrictedUnpickler(pickle.Unpickler):
def find_class(self, module: str, name: str):
allowed = _ALLOWED_MODULES.get(module)
if allowed is not None and ("*" in allowed or name in allowed):
return super().find_class(module, name)
if module.startswith("ktransformers"):
return super().find_class(module, name)
raise pickle.UnpicklingError(...)
Any module whose name starts with ktransformers is allowed in addition to the explicit list. The trust assumption is that no class anywhere in the ktransformers tree can be made into a deserialization gadget. The PR description does not enumerate the candidates. The auto code review attached to the PR did not either.
The KV-cache transport changes from mp.reductions.reduce_tensor (the function-call primitive on the client) to safetensors. The bidirectional function-call exposure on the client closes in the same diff.
At the bottom of the file, the patch swaps pickle.load(open(args.config)) to _safe_loads(f.read()). That closes a second deserialization vector: the config file. Anyone who can write the config file gets RCE on next process start. The CVE description does not mention this primitive. The PR description does not call it out either. It is in the diff because the author touched the file and saw it.
The PR has not merged. The repository's main branch carries the unfixed code. KTransformers is a 16,500-star LLM inference framework. The CVE was published April 23. The patch has been open since.
internal-only-by-convention, in the IPC channel between two processes that always live together
This is the internal-only-by-convention shape one floor below the network-header layer. CVE-2025-29927 was a Vercel "internal-only" HTTP header (x-middleware-subrequest) read with security-relevant authority on inbound network requests because the framework had no model for the network being hostile. CVE-2026-26210 is an "internal-only" ZeroMQ socket bound to every TCP interface because the codebase had no model for there being a network between its two processes at all. The legitimate client connects to localhost. The legitimate deployment runs --network=host. The "internal-only" contract is held by the absence of remote callers in the project's own code, and by the operator's understanding of which Docker flag they pasted.
grep -rn pickle.loads archive/ktransformers/server/balance_serve/ returns nine call sites. The patch addresses one. The other eight, in distributed/utils.py, distributed/parallel_state.py, distributed/custom_all_reduce_utils.py, plus the SchedulerClient response path, all deserialize objects that crossed a process boundary. They are gated by the same convention that gated the ROUTER socket. The convention has now been broken once on the wire. It will be broken again, in a different file, the next time someone routes a "trusted" boundary through TCP.
The same convention sits underneath most production Python ML stacks. torch.save uses pickle. torch.distributed's object collectives use pickle. multiprocessing uses pickle for queue traffic. The Python ML ecosystem assumes the boundary between processes is also the boundary between trust domains, because for a long time those processes only ever existed inside the same training job. KTransformers is what happens when that assumption ships as an inference server: the single-host research codebase grows a TCP socket and inherits the trust model of the single-host research codebase it grew out of. The next CVE in this class will not be in this file. It will be in whatever ML framework next exposes a pickle-shaped IPC channel to operators who paste --network=host from the README.
PoC and disclosure: chocapikk.com/posts/2026/ktransformers-pickle-rce. Patch: kvcache-ai/ktransformers#1944.
The fix did not remove the all-interfaces bind. It added a config key with a localhost default.