Guide · Code execution

How do I build a self-hosted code interpreter for my LLM?

Updated 2026-10-31

Run each piece of model-written code in a sealed machine you control: a VM with no network, an input folder shared read-only, an output folder it can write, CPU, memory and time limits, and a clean state restored after every run. With kosa8 that is one sandbox created with --egress deny, a Python container run inside it, and a checkpoint you rewind to. Paired with a local model, the whole loop runs without an internet connection.

01Steps

Install kosa8 and start its daemon. On an Apple silicon Mac or x86-64 Linux. The first run fetches the guest image and checks it against the release's published checksum.
terminal
```
curl -fsSL https://kosa8.com/install.sh | sh
kosa8 daemon start
```
Make an input and an output folder. The interpreter reads code and data from one folder and writes results to another. Nothing else on your computer is shared.
terminal
```
mkdir -p ~/interp/in ~/interp/out
```
Create a sealed sandbox. Two vCPUs, 2048 MiB of memory, no network at all (--egress deny blocks DNS too), the input folder read-only at /in and the output folder writable at /out. Containers in the sandbox see the same mounts.
terminal
```
kosa8 sandbox create interp --cpus 2 --memory 2048 --egress deny \
    --mount ~/interp/in:/in:ro --mount ~/interp/out:/out
```
Pull an image and save a clean checkpoint. The kosa8 service fetches the image on your side, so the sandbox needs no network for it. The checkpoint captures memory, processes and disk.
terminal
```
kosa8 sandbox pull interp python:3.12-slim
kosa8 checkpoint interp --label clean
```
Run code with a time limit. Write the model's code to the input folder and run it in a container inside the sandbox. timeout stops it after 30 seconds; whatever it writes to /out appears in ~/interp/out.
terminal
```
kosa8 sandbox run interp python:3.12-slim timeout 30 python /in/main.py
```
Reset after every run. Rewind puts the whole machine back to the clean checkpoint in about a second. It does not empty the shared output folder, so clear that yourself.
terminal
```
kosa8 rewind interp --to clean -y
```
Serve a local model for fully offline use. kosa8 model serve answers OpenAI- and Anthropic-style requests on 127.0.0.1:7434, so the model, the code and the results stay on the machine. Pull once while online.
terminal
```
kosa8 model pull Qwen/Qwen2.5-0.5B-Instruct-GGUF:q4_k_m
kosa8 model serve qwen2.5-0.5b-instruct-q4_k_m
export OPENAI_BASE_URL=http://127.0.0.1:7434/v1
```

02What does a code interpreter sandbox need?

An isolation boundary the code cannot reach past, no network unless the task needs it, no secrets inside, CPU, memory and time limits, a defined way to pass files in and results out, and a way back to a clean state after every run.

Boundary. A kosa8 sandbox is a virtual machine with its own kernel, so code that exploits a kernel bug lands in the VM, not your host.
Network. --egress deny is enforced on the host, so root inside the sandbox cannot lift it. If the code must install packages, use --egress-allow pypi.org --egress-allow files.pythonhosted.org instead.
Limits. --cpus, --memory (MiB) and --disk (GB) cap the machine; timeout caps each run.
Files. Mounts are HOST:GUEST[:ro]. Keep inputs read-only so a run cannot change the data the next run reads.
Clean state. Checkpoint and rewind restore memory, processes and disk; kosa8 sandbox rm interp deletes everything.

03How does the LLM call the interpreter?

Expose one tool, such as run_code, to the model. Your program, running on the host, writes the code to the input folder, runs it in the sandbox with a time limit, returns the trimmed output to the model, and rewinds the sandbox before the next call.

A minimal version in Python, using only the commands above:

run_code.py

import pathlib, subprocess

IN = pathlib.Path.home() / "interp/in"

def run_code(code: str) -> str:
    """Run model-written Python in the sealed sandbox, then reset it."""
    (IN / "main.py").write_text(code)
    try:
        r = subprocess.run(
            ["kosa8", "sandbox", "run", "interp", "python:3.12-slim",
             "timeout", "30", "python", "/in/main.py"],
            capture_output=True, text=True, timeout=60)
        out = r.stdout + r.stderr
    except subprocess.TimeoutExpired:
        out = "timed out"
    subprocess.run(["kosa8", "rewind", "interp", "--to", "clean", "-y"],
                   capture_output=True)
    return out[-10000:]

Register run_code as a tool with any OpenAI-compatible client pointed at OPENAI_BASE_URL. Trimming the output keeps a runaway print loop from filling the model's context.

04Can a self-hosted code interpreter run offline?

Yes. The sandbox runs with --egress deny, which blocks all traffic including DNS, and a model served by kosa8 model serve answers on 127.0.0.1:7434, so prompts, code and results never leave the machine. Once the guest image, the Python image and the model have been fetched, no internet connection is needed.

For a machine that is never online, kosa8 reads KOSA8_OFFLINE=1 and fetches nothing: copy the guest image in from an air-gap bundle instead.

05Can it keep state between cells like a Jupyter kernel?

Partly. Keep one container running in the sandbox with kosa8 sandbox start and send each cell to it with kosa8 sandbox exec: files and installed packages persist between cells. Python variables persist only if a process inside keeps running, such as a Jupyter kernel from an image that already includes it.

kosa8 sandbox start interp python:3.12-slim sleep infinity prints a container id; kosa8 sandbox exec interp ID python /in/cell.py runs a cell in it. Take a checkpoint between cells to step back one cell at a time; kosa8 keeps ten per sandbox.

06Other options

Docker with --network none, memory and CPU limits and a non-root user is a reasonable floor for code you mostly trust. Containers share the host kernel.
gVisor runs containers on a user-space kernel on Linux.
Firecracker or Kata Containers are microVM building blocks for Linux hosts with KVM, suited to a multi-tenant execution service you run yourself.
E2B, Daytona or Modal are hosted sandboxes you call over an API, a good fit when code should run in the cloud.
Framework executors such as smolagents' Docker executor or the OpenAI Agents SDK's Docker sandbox can point at kosa8's Docker-compatible socket.