Fusefile
The declarative spec format for a Fuse environment
A Fusefile is the canonical, human-authored way to describe a Fuse environment: what image or rootfs to use, how much CPU/memory/storage/GPU it needs, what build and run commands to execute, what services to bring up alongside it, and what ports to expose.
A Fusefile is parsed and compiled entirely client-side (by the CLI) into the
orchestrator’s wire format. The orchestrator itself never sees a Fusefile, only the
compiled CreateEnvironmentRequest.
Everything below is the complete field reference. If you have not written one before, start with Your first Fusefile, which is the smallest file that boots, explained line by line.
Editor support
Fuse publishes a JSON Schema for the Fusefile. Point your editor at it with a
yaml-language-server modeline on the first line and you get completion,
hover documentation, and inline validation as you type:
# yaml-language-server: $schema=https://raw.githubusercontent.com/folsomintel/fuse/main/schema/fusefile-v1.json
version: 1
resources:
cpus: 2
memory: 2GB
run: ./start.sh
The modeline is a YAML comment, so it changes nothing about how the file is
parsed. fuse init writes it into every scaffolded Fusefile.
The schema works in any editor that speaks the YAML Language
Server: VS Code with the YAML
extension, Neovim with yamlls, JetBrains IDEs, and Helix. In an editor that
lets you map schemas to file names instead, point the same URL at Fusefile.
One catch: because a Fusefile has no extension, most editors do not recognize
it as YAML at all, so you get neither highlighting nor the schema until you
associate the filename. Editor setup has the
config for VS Code, Zed, Neovim, Vim, Helix, JetBrains, Emacs, and bat.
Two things to know about it:
- It is advisory.
fuse upstill parses, validates, and compiles the file, and the orchestrator validates the compiled request independently. A green editor is not a promise thatfuse upsucceeds. Runfuse validatefor the authoritative answer. - Go is the source of truth. If the schema and the CLI disagree, the CLI is right and the schema is the bug. Please open an issue.
The schema is versioned by filename. A future format version ships as a new file at a new URL rather than changing this one, so a modeline you commit today keeps describing the format you wrote against.
Full field reference
# yaml-language-server: $schema=https://raw.githubusercontent.com/folsomintel/fuse/main/schema/fusefile-v1.json
version: 1
# the task id this file boots under, and with it the environment's name: the
# orchestrator prefixes it, so this is the environment `fuse-sandbox`. omit it
# and the cli falls back to the parent directory's name.
name: sandbox
# base rootfs to boot, by name. NOT an oci ref -- the host agent resolves it
# against the rootfs images baked on that host. omit for the host's default.
image: worker-base
# files materialized in the guest before the build steps run. config and small
# code only; the combined size is capped at 64KiB.
files:
- path: config/app.yaml # relative paths resolve against the workspace
source: ./app.yaml # read relative to this Fusefile, not the cwd
- path: entrypoint.sh
content: | # or inline the body instead of naming a source
#!/bin/sh
echo hello
mode: "0755" # optional octal mode
# local files and directories shipped into the guest, also before the build
# steps. this is the one that takes a directory. combined size capped at 512KiB.
copy:
- from: ./start.sh # relative to this Fusefile, not your cwd
to: ./start.sh # relative paths resolve against the workspace
- from: ./src # a directory is walked into one upload per file
to: /workspace/src
resources:
cpus: 2 # whole vCPUs; 2.0 is accepted, a fraction is not
memory: 2GB # M/MB/MiB, G/GB/GiB, T/TB/TiB (all binary), compiles to ram_mb
disk: 10GB # same units, rounded up to whole GB; 'storage' is an alias
region: us-east-1 # only schedules onto hosts registered in this region
max_runtime: 1h # hard ttl from create; a leak ceiling, not a budget
idle_timeout: 15m # destroyed after this long with no exec and no attach
# advanced, and only on a qemu-backed host with NVIDIA hardware. omit all
# three unless you need a GPU. see "Advanced: GPU fields" below.
# gpu: 1
# gpu_kind: a100
# gpu_profile: 1g.10gb
# hard placement constraints for a self-hosted fleet. every field is a gate,
# not a preference: a request that matches no host is rejected, never queued.
placement:
host: build-3 # exact host id. still has to clear the other gates.
labels: # every pair must match the host's declared labels (AND).
disk: nvme
tier: build
# opt in to the build layer cache. off by default.
cache:
enabled: true
# environment variables for every build step and for run. same value-or-secret
# grammar as services.<name>.env. services do not inherit these.
env:
NODE_ENV: { value: production }
PORT: { value: "8080" }
DATABASE_URL: { secret: db_url } # requires db_url, listing it below is optional
# work that prepares the environment. runs to completion before run, and
# compiles into startup_script ahead of it. `setup:` is the old name for this
# block and still works; setting both is an error.
build:
# bare string form: one step, keyed on its bytes plus the step before it.
- apt-get update -qq && apt-get install -y --no-install-recommends ripgrep
# mapping form: `inputs` are hashed into the step's cache key.
- run: npm ci
inputs:
- package.json
- package-lock.json
# `workdir` scopes one step to a directory. relative paths resolve against
# the workspace, and the change does not leak into the next step.
- workdir: web
run: npm run build
# a step that reads secrets or has effects outside the rootfs must opt out.
- run: ./scripts/register-with-vault.sh
cache: false
# services brought up inside the vm. compiles to manifest.services then a compose project.
services:
postgres:
image: postgres:16
ports: [5432]
env:
POSTGRES_PASSWORD: { secret: pg_password }
redis:
image: redis:7
ports: [6379]
# bound on build + run. default 30s, ceiling is an operator setting (55s out of
# the box). headroom for a slow build, not a budget for a long one.
startup_timeout: 55s
# the main task entrypoint, compiled into startup_script (after build). a plain
# string is interpreted by sh -lc; a list ["python", "app.py"] is an argv whose
# elements are shell-quoted, so spaces, quotes, $, and globs in an argument
# cannot alter the command. use the list form only when an argument would
# otherwise be reinterpreted by the shell.
run: ./start.sh
# absolute path, created with `mkdir -p`. the build steps and run execute here.
# defaults to /workspace when omitted. see Workspace below for what does not
# inherit it.
workspace: /workspace
# ports published to the outside world (ingress).
expose:
- port: 8080
as: http
# environment-level readiness probe: one verdict for whether the whole sandbox
# is doing its job, reported back to the control plane. exactly one of http and
# exec. see Healthcheck below.
healthcheck:
http:
port: 8080
path: /healthz
# exec: ["/app/ready", "--check"]
interval: 5s # how often to re-check
timeout: 2s # bound on one attempt; must not exceed interval
retries: 12 # consecutive failures that flip the verdict to failing
start_period: 10s # grace window before failures are counted
# secret names this environment requires. values are supplied out-of-band
# (cli flag / env / secret store), never written in the Fusefile.
secrets:
- pg_password
Field notes
These are the fields a typical environment uses. The three GPU keys are deliberately not in this table: they only apply on a QEMU-backed host with NVIDIA hardware, and they are covered separately under Advanced: GPU fields. If you are not requesting a GPU, you can stop reading at The setup layer cache.
| Field | Type | Notes |
|---|---|---|
version |
int | Currently always 1. A Fusefile is a single yaml document; anything after a second --- is rejected rather than silently dropped. |
name |
string | The task id this Fusefile boots under, and through it the environment’s identity: the orchestrator prefixes it, so name: sandbox is the environment fuse-sandbox. Optional. A DNS label: lowercase letters, digits and dashes, alphanumeric at both ends, 63 chars max. See Naming an environment. |
image |
string | Name of a rootfs pre-baked on the host, not an OCI image ref. There is no OCI pull: fc-agent resolves it to <IMAGES_DIR>/<image>.ext4 and qemu-agent to <IMAGES_DIR>/<image>.qcow2, and returns a 400 if no such file exists. An operator bakes and places it out of band (e.g. with fc-bake-rootfs.sh). Omit to use the baked base rootfs. See Base images. |
files |
list of {path, source | content, mode} |
Files written into the guest before the build steps run. See Files. |
copy |
list of {from, to} |
Local files and directories copied into the guest before the build steps run. See Copy. |
copy[].from |
string | A file or directory on your machine, resolved relative to the Fusefile, not your working directory. A directory is walked into one upload per regular file. A symlink is an error, never followed. |
copy[].to |
string | Where it lands in the guest. Absolute, or relative to workspace. .. segments are rejected, as is anything under /fuse, which holds the guest agent’s own manifest, secrets and credentials. Two entries may not write the same path. |
resources.cpus |
int | Whole vCPU count. A whole-valued float (2.0) is accepted; a fraction (0.5) is rejected, because Firecracker takes an integer vcpu_count and QEMU an integer -smp. A negative count is rejected. Omitting it (or 0) means “host default”: the host agent boots the VM with 1 vCPU. |
resources.memory |
string | A size, e.g. 2GB. Compiles to ram_mb. See Size units. |
resources.disk |
string | Size of the guest root disk, e.g. 10GB. Compiles to storage_gb, rounded up to a whole GB, so 512MB provisions 1GB. See Size units. |
resources.storage |
string | A permanent alias for resources.disk; disk is the preferred spelling. Setting both is only valid if they name the same size. |
resources.region |
string | Schedules only onto a host registered in this region (fuse host register --region). Empty matches any region. |
resources.max_runtime |
string | Go duration (1h, 30m); compiles to max_runtime_seconds. A leak-detection ceiling measured from create, not a task budget. See Reconciliation. |
resources.idle_timeout |
string | Go duration, minimum 1m; compiles to idle_timeout_seconds. Destroys the environment once it has gone this long with no exec and no attach session. Omit for no idle expiry. |
placement.host |
string | Pins the environment to an exact host id. A pin is not an override: the host must still be active, run the right backend, and fit the request. An unknown host id fails fast with a 404. |
placement.labels |
map | Label selectors matched against the labels an operator declared with fuse host register --label key=value. Every pair must match (AND). A selector that matches no host is rejected immediately. |
cache.enabled |
bool | Opt in to the build layer cache. Off by default. |
env |
map | Environment variables set for every build step and for run. Services do not inherit them, and neither do fuse environment exec and fuse environment shell. See Environment variables. |
env.* |
literal or secret ref | Exactly one of value or secret must be set per entry, the same grammar as services.*.env.*. The name must be a shell identifier ([A-Za-z_][A-Za-z0-9_]*); an empty or malformed name is rejected. { secret: name } requires that secret, so listing it under secrets is optional. |
build |
list of strings or steps | Steps that prepare the environment. They run once at boot, to completion, before run, and compile into startup_script ahead of it. Each entry is either a bare string or a {run, inputs, cache, workdir} mapping. See Build and run. |
build[].run |
string | The step’s shell command. Required in the mapping form. |
build[].inputs |
list of strings | Files (or directories, or globs) whose contents are hashed into the step’s cache key. Relative to the Fusefile’s directory; absolute and ..-traversing paths are rejected. |
build[].cache |
bool | Set false to make the step uncacheable. Required for a step that reads secrets or has effects outside the rootfs. |
build[].workdir |
string | Directory this one step runs in. A relative path resolves against workspace; an absolute path is taken as-is. .. segments are rejected. The step is emitted as a subshell, so the directory does not leak into the next step. See Workspace. |
setup |
list of strings or steps | Deprecated alias for build, kept accepted so every Fusefile written before the rename keeps working unchanged. It takes the same entries and compiles to the same bytes. Setting both build and setup is an error. |
services |
map | In-VM services, each with image, ports, and env. Compiles to manifest.services plus a compose project. |
services.*.ports |
list of ints | Container ports, each between 1 and 65535. |
services.*.env.* |
literal or secret ref | Exactly one of value or secret must be set per entry. The variable name must not be empty. { secret: name } resolves from the secrets you supply at up time. |
run |
string or list of strings | The main entrypoint, compiled into startup_script after build. A string is interpreted by sh -lc; a list is an argv, each element shell-quoted into a single command line so no argument is subject to word splitting, globbing, or metacharacter reinterpretation. The list form is not a shell-less execve: the generated command still runs inside sh -lc, so it protects arguments from shell interpretation rather than removing the shell. Use it only when an argument would otherwise be reinterpreted (spaces, quotes, $, globs). |
startup_timeout |
string | Go duration bounding build + run. Omit for the orchestrator default (30s). See Startup timeout. |
workspace |
string | Working directory for the build steps and run. Defaults to /workspace. Must be an absolute path with no .. segments; the generated script mkdir -ps it and cds into it. Services, fuse environment exec, and fuse environment shell do not inherit it. See Workspace. |
expose |
list of {port, as} |
Guest ports published externally. Each port is between 1 and 65535 and may appear only once. as is an optional name, unique across the list, in DNS-label form (lowercase letters, digits and dashes, alphanumeric at both ends, 63 chars max). |
healthcheck |
{http | exec, interval, timeout, retries, start_period} |
The environment-level readiness probe. Exactly one of http and exec. Omit the whole block for no probe. See Healthcheck. |
healthcheck.http |
{port, path} |
An HTTP GET inside the guest; a 2xx or 3xx response passes. port is between 1 and 65535 and needs no matching expose entry, because the probe runs in-guest. path must start with / and defaults to /. |
healthcheck.exec |
list of strings | An argv run inside the guest; exit status 0 passes. An argv rather than a shell string, so no element can be reinterpreted as a pipeline, a redirect, or a glob. |
healthcheck.interval |
string | Go duration between attempts. Omit for the guest agent’s default (10s). |
healthcheck.timeout |
string | Go duration bounding one attempt. Omit for the guest agent’s default (2s). Attempts run one at a time, so a value above interval is rejected rather than silently stretching the interval. |
healthcheck.retries |
int | Consecutive failures that flip the verdict to failing. Omit for the guest agent’s default (3). A single failed attempt is never a verdict. |
healthcheck.start_period |
string | Go duration measured from the first attempt, during which failures are not counted. It ends for good once the probe passes once. |
desktop |
{width, height} |
The geometry of the environment’s graphical session, both required, 320 to 3840 each. Requires an image baked with the desktop stack; on any other image the block is inert. Omit the whole block to keep a desktop image’s baked default. See Desktop. |
secrets |
list of strings | Secret names this environment requires. An empty name is rejected: nothing you could pass to --secret would satisfy it. Values are supplied out-of-band via --secret key=value or --secrets-file, never written into the Fusefile itself. |
Naming an environment
An environment’s id is its task id with the orchestrator’s prefix on the front,
so a task id of sandbox is the environment fuse-sandbox. That is the name
you pass to fuse environment exec, fuse snapshot create, and everything
else that addresses a running environment.
Three things can supply the task id, in order:
fuse up --task-id <id>, which always wins.name:in the Fusefile.- The Fusefile’s parent directory name, as a fallback.
Only the third is a guess, and it is the only one up announces. It is also
the one that bites: two checkouts of the same repo derive the same id, so the
second fuse up fails with task already assigned. Declaring name: makes
the id a property of the project instead of of wherever the file happens to
sit.
version: 1
name: sandbox
run: ./start.sh
$ fuse up
# -> environment fuse-sandbox
$ fuse environment exec fuse-sandbox -- uname -r
The collision is deliberate. Because the name is stable, a second fuse up
against a live environment is refused rather than quietly starting a duplicate
you would have to find and clean up. To run several at once, give each one its
own id with --task-id.
Build and run
build and run are two phases of one boot. Every build step runs to
completion first, in the workspace, and only then does run start. Both
compile into the single startup_script the orchestrator executes during
create.
They are emitted as two distinguishable halves rather than one flat
concatenation. The script reaches the guest as a single sh -lc with both
output streams discarded, so its exit status is the only thing that comes back,
and a build failure has to be told apart from a task failure:
set -eu
if (set -o pipefail) 2>/dev/null; then set -o pipefail; fi
mkdir -p '/workspace'
cd '/workspace'
# fuse: build phase (a failure here exits 90)
trap '[ $? -eq 0 ] || exit 90' EXIT
npm ci
trap - EXIT
# fuse: run phase
./start.sh
A failing build step exits 90, whatever status the step itself returned. The
trap is cleared before run, so a failing run still reports its own status
unchanged. A Fusefile with no build steps has only one phase, so it carries no
markers and compiles to exactly the script it always did.
Nothing here makes build cacheable on its own: the steps rerun on every boot
unless you opt into the layer cache or bake them into
an artifact with fuse build.
setup is the old name for build
setup: is a deprecated alias, kept accepted so every Fusefile written before
the rename keeps working. It takes the same entries, derives the same layer
keys, and compiles to the same bytes:
build:
- npm ci
# the same file, in the old spelling
setup:
- npm ci
Setting both is an error rather than a merge. They are one field under two names, so concatenating them would run steps in an order nobody wrote, and picking a winner would silently drop half the file.
Size units
memory, disk, and storage all take the same grammar: a number, optional
whitespace, and a unit. The unit is one of M, MB, MiB, G, GB, GiB,
T, TB, TiB, in any case. All of these are valid and mean the same thing:
memory: 2GB
memory: 2G
memory: 2 GB
memory: 2GiB
memory: 1.5GB # 1536 MiB
memory: 512MB
Every unit is binary. GB and GiB are synonyms for 1024 MiB, and 2GB
has always compiled to ram_mb: 2048, which becomes Firecracker’s
mem_size_mib and QEMU’s -m. Redefining GB as decimal would be more
correct and would silently shrink every existing memory: 2GB by 7 percent, so
it is not done.
Two things are rejected rather than guessed at:
- A bare number (
memory: 2048) has no unit and is genuinely ambiguous. - A size that does not land on a whole MiB (
memory: 0.25MB) is an error instead of being truncated down to less memory than you asked for.
disk is then rounded up to a whole GB on the way to storage_gb, so
disk: 512MB provisions 1GB. It is accounting-only either way: the scheduler
uses it to decide whether a host fits, and no host agent resizes the root disk
to match.
Lifetime fields
max_runtime and idle_timeout answer different questions and neither replaces
the other.
max_runtime is a ceiling measured from create. The reconcile loop uses it to
find leaks: a VM older than its ceiling that still carries a task is destroyed. It
is not a budget and not a health check, so making progress does not spare a VM and
being wedged does not doom one. Set it above any plausible healthy runtime.
idle_timeout is a window measured from the last activity. An environment is
idle when nobody has exec’d into it and no attach session is open. Once the window
elapses the environment is destroyed. There is no pause or suspend anywhere in the
stack, so idle expiry means teardown, not scale-to-zero.
Activity means exactly two things: an exec call, and an open attach session. An
attach session counts for as long as it stays open, however quiet it is, and the
window restarts when it closes. In-guest CPU use and traffic on exposed ports are
not observed, so a VM computing hard with nobody attached will still be torn
down once its window elapses. If that is your workload, do not set idle_timeout.
Detection runs on the reconcile loop (30s ticks) and requires two consecutive
observations, so real teardown lands roughly one to two ticks after the window
elapses. That is why the minimum accepted value is 1m.
Healthcheck
Without a healthcheck: block, an environment reaching running means one
thing: a process was started on the host and the startup script returned.
Nothing dialled your app. A guest whose workload panicked stays running until
it trips max_runtime, because the only liveness signal the control plane has
is that the VM still exists.
healthcheck: is the block that closes that gap. It declares one probe, whose
one verdict answers “is this sandbox doing its job”.
healthcheck:
http:
port: 8080
path: /healthz
interval: 5s
timeout: 2s
retries: 12
start_period: 10s
Exactly one of http and exec is set. Two probes would produce two verdicts,
and the point of the block is that there is one:
healthcheck:
exec: ["/app/ready", "--check"]
The probe runs inside the guest, in the guest agent. An http probe dials
127.0.0.1:<port> there, so the port needs no expose entry: the question is
whether the app is serving, not whether anything outside can reach it. An exec
probe runs the argv directly with no shell.
How a verdict is reached
The verdict is one of three values:
| Verdict | Meaning |
|---|---|
starting |
The probe has not passed yet and is still inside start_period, so failures are not being counted. |
passing |
The most recent attempt succeeded. |
failing |
The probe failed retries times in a row after start_period ended. |
A single failed attempt is never a verdict, which is what retries is for. A
success resets the failure count and clears the last error, so a passing
environment never carries stale detail from before it recovered.
start_period is a grace window on failures only, measured from the first
attempt. It ends for good on the first success: an app that came up and then
fell over is failing, not still starting.
Every timing you omit is filled in by the guest agent, not by the Fusefile
compiler or the orchestrator, so there is exactly one place each default lives.
fuse compile prints omitted ones as (guest default) rather than as 0.
What it does and does not do
The verdict is reported, not acted on. It surfaces on the environment
(fuse environment get, the health field on every SDK’s environment type)
and it is what fuse up --wait-healthy
waits for. That is all it does:
- It is not a lifecycle state.
statestaysrunningwhatever the probe says, becausestate’s vocabulary is a closed set every SDK reasons about and an unhealthy environment is still a running one. - Nothing is destroyed for a failing probe. The only destructive predicates in reconciliation remain the age and idle ceilings. Report first, act later.
- The orchestrator reads the verdict back on its reconcile tick (30s by default), so what you read lags the guest by up to a tick.
Not the same as services.*.healthcheck
The two blocks share a name and answer different questions.
services.<name>.healthcheck is compose-native. It governs one container, is
evaluated by the guest’s docker compose, and a container it marks unhealthy is
unhealthy inside the guest: nothing about it reaches the orchestrator.
The top-level healthcheck: is about the environment as a whole, is evaluated
by the guest agent, and its verdict travels back to the control plane. Both can
be set on the same Fusefile; they do not interact.
Desktop
image: desktop
desktop:
width: 1280
height: 800
resources:
memory: 4GB # a browser session does not fit the 1GB default
desktop: sets the geometry of the environment’s graphical session, for
environments meant to be driven by a computer-use agent. It requires an image
baked with the desktop stack (see the
desktop environments guide); on any other
image the block is inert and the computer surface reports the display as
absent.
Both dimensions are required. A guessed dimension would silently shift every coordinate a computer-use model emits, which presents as model failure rather than as the config error it is — so there is deliberately no default to fall back to. Omitting the whole block is different: that keeps the desktop image’s baked default (1024x768).
The geometry travels to the guest as /fuse/desktop.json, and the guest agent
restarts the display at the declared size if it came up at a different one.
The display number is fixed at :1.
Base images are not OCI images
image names a rootfs that has already been baked onto the host, and the host
agent resolves that name against the images stored there. It is not a container
reference, and there is no registry pull: image: ghcr.io/acme/worker:latest
fails the create with base image not found.
The distinction is per-field, not global. services[].image is an OCI
reference, because services run as containers inside the guest via podman and
compose. Only the top-level image, which decides what disk the VM boots from,
is a host-local rootfs name.
To get a custom base, either bake a rootfs on the host (fc-bake-rootfs.sh, or
qemu-bake-cuda-rootfs.sh for GPU hosts) or run
fuse build, which runs your build phase once and
snapshots the result into a bootable artifact.
Workspace
workspace is the directory the build steps and run execute in. It defaults to
/workspace, and it must be an absolute path with no .. segments. The
compiled startup script creates it and moves into it before anything else runs:
mkdir -p '/srv/app'
cd '/srv/app'
The path is shell-quoted, so a space or a quote in it is safe. A relative
files[].path resolves against the workspace, and so does a relative
build[].workdir.
Scoping one step to a directory
A build step written as a mapping can set workdir, which runs that step
somewhere else without disturbing the others:
workspace: /srv/app
build:
- npm ci # runs in /srv/app
- workdir: web # runs in /srv/app/web
run: npm run build
- ./scripts/package.sh # back in /srv/app
run: node server.js
The step compiles to (cd 'web'; npm run build). The subshell is what keeps
the directory scoped, so the next step still starts in the workspace.
Only the directory is scoped. All build steps share a single shell, so
set -eu, exported variables, and any shell options a step sets still carry
across into the steps after it. If you need a variable to stay local to one
step, scope it yourself.
What does not inherit the workspace
- Services. A service runs in its own container, so
workspaceis a guest path and not a path inside the service’s image at all. Inheriting it would point the container at a directory that does not exist there, so this is deliberate rather than unimplemented. fuse environment execandfuse environment shell. Both start in the guest’s home directory.cdinto the workspace yourself, or pass a full path.
manifest.machine.workspace is present in the compiled manifest and carries
the same value, but it is reserved: no component reads it today. The
mkdir -p and cd in the startup script are what actually put you in the
workspace.
Files
files writes files into the guest before the build steps run, so build and run can
read config, scripts, and small source trees that live next to the Fusefile
rather than being fetched at boot.
files:
- path: config/app.yaml
source: ./app.yaml
- path: entrypoint.sh
content: |
#!/bin/sh
exec python -m app
mode: "0755"
Each entry sets exactly one of source (a path on your machine, resolved
relative to the Fusefile, not your working directory) and content (an inline
literal). path is where it lands in the guest; a relative one resolves against
the workspace. mode is an optional octal string.
Entries compile into the startup script as base64 blocks, so content is carried verbatim: binary data, quotes, and shell metacharacters all survive, and nothing in a file is expanded by the shell.
Files are rewritten on every boot, including a fuse up --from-build boot that
skips the build steps, since they are authoring inputs that may have changed since the
artifact was baked.
Copy
copy is the directory-capable sibling of files. Each entry names a from on
your machine and a to in the guest, and everything lands before setup runs:
copy:
- from: ./start.sh
to: ./start.sh
- from: ./src
to: /workspace/src
- from: ../shared/config.yaml
to: /etc/app/config.yaml
setup:
- chmod +x ./start.sh
run: ./start.sh
The rules:
fromis relative to the Fusefile, not to your working directory, sofuse up ./repro/Fusefilecopies the same sources it would from inside./repro. An absolute path is taken as written.tomay be absolute or relative toworkspace, the same directorysetupandrunalready start in. A relativetowith the default workspace means/workspace/<to>.- A directory is expanded on your machine, one upload per regular file,
landing under
towith its structure preserved. Empty directories do not travel: the guest creates a file’s parents when it is written. - Symlinks are an error, not a silent follow. Following one would ship whatever it points at, which for a link out of the tree is a file you never meant to send. Name the target instead.
- Nothing may land under
/fuse. That is where the guest agent keeps its manifest, its resolved secrets, and its TLS credentials. The CLI rejects it, and so does the orchestrator.
Choose between the two blocks by what you have: files for a literal body or a
mode, copy for a directory or for a file you would rather not restate.
Two other limits worth knowing before you reach for it:
- Permissions are not preserved. The upload carries a path and a body and
nothing else, so an executable arrives without its executable bit.
chmodit insetup, as the example above does. - A fork does not re-copy.
fuse environment forkstarts from a copy of the source’s disk, so it already has the files. A restored environment does get them again, since it may have been edited since the snapshot.
.fuseignore
copy: {from: .} on a real checkout would ship .git/, node_modules/ and
.env if nothing stopped it. A .fuseignore next to the Fusefile is what
stops it, and a set of defaults is applied whether or not you write one:
.git/ node_modules/ __pycache__/ .venv/ venv/ target/ dist/ build/
.DS_Store *.pyc .env .env.* *.pem *.key
The first group is size. The last four are secrecy: a Fusefile names its
secrets without carrying their values, so a copy that quietly uploaded .env
or a private key would undo that on its first use.
The syntax is gitignore’s:
| Pattern | Matches |
|---|---|
*.log |
that name at any depth, so both app.log and logs/app.log |
build/ |
directories only, never a file called build |
/tmp-scratch |
only at the top of the copy source, not src/tmp-scratch |
docs/drafts |
the same: a / anywhere in a pattern anchors it |
build/**/*.o |
** spans any number of directories, * stops at one |
!keep-this.log |
re-includes something an earlier pattern dropped |
# a comment |
nothing. Blank lines are skipped too |
The last pattern that matches decides, and the defaults are matched before anything in your file. That is what makes overriding one a single line:
# yes, I really do want the env file in the guest
!.env
Four things worth stating outright:
- An ignored directory is pruned, not filtered. Nothing under it is walked,
read, or counted, which is why
.fuseignorecan bring a tree back under the 512 KiB cap rather than just hiding part of it from the guest. - Ignores only bound a directory source. A
fromthat names one file copies that file,.envincluded: you asked for it by name. - Patterns anchor to the copy entry’s source, which for the usual
from: .is the Fusefile’s directory. Withfrom: ./src,/main.gomeanssrc/main.go. - Only the one file next to the Fusefile is read. Nested
.fuseignorefiles are not, and neither is.gitignore: it excludes build outputs an environment often does need, and reading it would make a Fusefile’s behavior change when someone edits a file that has nothing to do with it.
To see what all of this leaves, run
fuse up --show-copy, which prints
each entry’s file count and size, and how much was dropped by your patterns
versus by the defaults, before anything is created.
Environment variables
env sets variables for every setup step and for run. It takes the same
value-or-secret entries as services.*.env, so there is one env shape in the
file:
env:
NODE_ENV: { value: production }
DATABASE_URL: { secret: db_url }
setup:
- npm ci
run: node server.js
A name must be a shell identifier: a letter or underscore followed by letters,
digits and underscores. NODE-ENV and 2FAST are rejected at compile time.
Referencing a secret requires it, the same way a service’s env ref does, so
db_url above does not also have to appear under secrets. A missing value
fails fuse up before anything is created.
Values are never written into the startup script
The compiled startup script reaches the guest as a single sh -lc argument,
which means its text is visible in the host’s process table for as long as the
boot takes. So a secret is not interpolated into it. The compiled manifest
carries the env block instead, the orchestrator resolves each reference and
writes the result to /fuse/env in the guest, and the script sources that path:
set -a
. /fuse/env
set +a
Only the path is ever in the script. /fuse is mounted on tmpfs in the guest,
so the resolved values never reach persistent storage either. set -a is what
exports them, so setup, run, and anything they spawn all inherit them.
What does not see them
- Services. A service gets its environment from its own
services.*.envblock. Inheriting the machine-wide one would push values into a container that never asked for them, so it is deliberate rather than unimplemented. fuse environment execandfuse environment shell. Both reach the guest over SSH without sourcing anything, so a variable set here is scoped to the boot-time script. Source/fuse/envyourself if you need it in a shell.fuse build. The setup phase runs there through the exec path, and layer keys are derived from the setup fragments alone, so build scripts do not source the env file. A step that needs a value during a bake should read it from/fuse/secrets.jsonwithcache: false.
Startup timeout
build and run compile into one startup_script that the orchestrator runs
synchronously inside the create request. startup_timeout bounds it:
startup_timeout: 55s
Omit it and the orchestrator’s default (30s) applies. Exceed it and the create
fails with startup script did not complete in time.
Because the script runs inside the request, the bound is capped by an operator
setting (-max-startup-script-timeout, 55s by default, which must stay under
the server’s -write-timeout). Asking for more than the ceiling is refused with
a 400 naming the maximum, rather than being silently clamped to it.
So this field is headroom for a build phase that is a little slow, not a budget for one that is genuinely long. A build that installs packages or pulls a model will not fit under any allowed value. Move that work into an image instead:
fuse build # runs the build steps once, on the 600s exec path
fuse up --from-build <id> # boots the result, skipping them entirely
The setup layer cache
build steps compile into one startup script that re-runs in full on every
fuse up. cache: {enabled: true} opts the Fusefile into content-addressed
layers, so a step whose inputs have not changed can be skipped instead of rerun.
Caching is opt-in: a layer is a rootfs captured mid-provisioning, so it is a
deliberate choice, never a default.
Each step gets a key derived from the step before it, which is what makes invalidation directional:
layer_key(i) = sha256(
"fusefile-layer/v2\n" +
parent_key(i) + "\n" + // layer_key(i-1); the base key for i == 0
step_script(i) + "\n" + // the step's `run` string, byte for byte
inputs_digest(i) + "\n" +
workspace) // the steps run after `cd <workspace>`
base_key = "image:" + sha256(<base image reference> + the `files` block)
inputs_digest = sha256 over the sorted list of
(relative path, mode & 0o111, sha256(content)); "" when unset
Edit step N and steps N..end all get new keys; steps before N are untouched. The same chaining is why a miss cascades: step N+1’s key is defined in terms of step N’s, so a step whose parent is unknown cannot be keyed at all, and neither can anything after it.
Deliberately not in the key: secret values, the task ID, run: (it is the
entrypoint, not a layer), and every resources field, since none of them change
the filesystem. There is no normalization either: a byte change in a step is a
miss, because guessing at semantic equivalence is how a cache serves a stale
rootfs.
The host architecture is not in the key either, but for a different reason than the rest. An ext4 rootfs genuinely is not portable across architectures, so architecture is a real constraint on serving a layer. It just is not knowable when the key is derived, because that happens before the build has been scheduled onto any host. Folding in a guess would label an artifact with an architecture it was not built on, and a later lookup would hit that key and be handed a rootfs it cannot boot. So it is recorded on the artifact by the host that actually built it, and lookups filter on it separately. See The layer cache.
A step that reads /fuse/secrets.json, or has effects outside the rootfs, must
set cache: false. Secrets are written to disk before the build steps run, so a layer
captured after such a step would bake secret material into a rootfs keyed on
content that excludes those secrets, and a later up with rotated values would
get a hit. Declaring inputs on an uncacheable step is an error.
One hard limit: a GPU environment gets no caching at all, because GPU
environments run on the qemu backend, whose snapshot endpoints hard-501 since a
vfio device cannot be checkpointed. That is worth knowing early, as those are
often the hosts with the most expensive build: phases.
Layers are stored on the host that built them, but they are no longer confined to it: a host that needs a layer it does not have fetches it directly from a host that does. See The layer cache for how that works and what it means for a multi-host fleet.
To see the derived keys without creating anything:
fuse up --plan
fuse build --plan
Both print the same plan: there is one key derivation, shared by the two commands
that run the build phase. --no-cache on either ignores the cache block for
one run.
fuse build is the more useful of the two, since it is
the command that runs build: on its own and captures the result. On a
fuse up --from-build boot, build: does not run at all,
so there is no layer plan for it and --plan and --no-cache are rejected
there rather than silently ignored.
--plan always reports every cacheable step as a miss. It deliberately does not
look anything up: resolving needs a target architecture, which needs the fleet,
and a command whose whole purpose is “show me what would happen” should not
require the orchestrator to be reachable. The real hit and miss breakdown is
reported by the build itself.
Advanced: GPU fields
Skip this section unless your workload needs a GPU. gpu, gpu_kind, and
gpu_profile are the only NVIDIA-specific fields in a Fusefile, they are all
optional, and requesting any of them is what routes the environment onto a host
registered with the qemu backend. A Fusefile with no gpu key keeps
scheduling onto the default Firecracker backend and never touches any of this.
| Field | Type | Notes |
|---|---|---|
resources.gpu |
int | A device count, never a fraction. Counts whole physical GPUs, or MIG instances when gpu_profile is set. Requires a host registered with the qemu backend. |
resources.gpu_kind |
string | Optional GPU model match, e.g. a100. Matched case-insensitively as a substring of the host’s reported device model, so a100 matches NVIDIA A100-SXM4-40GB. Empty matches any kind. See Providers. |
resources.gpu_profile |
string | Optional MIG profile, e.g. 1g.10gb. Asks for hardware-isolated fractions of a GPU instead of whole devices, and changes what gpu counts. See GPU workloads. |
gpu is a count of devices, not an amount of GPU
The one thing to get right here: gpu is always a whole number of things,
and gpu_profile decides what those things are.
resources:
gpu: 2 # two entire physical GPUs
resources:
gpu: 2
gpu_profile: 1g.10gb # two 1g.10gb MIG slices, which together are a fraction of one card
Both files say gpu: 2, and they ask for very different amounts of hardware.
There is no way to spell “half a GPU” as a fractional gpu value; a fraction is
requested by naming the profile you want and counting how many of it you need.
On an A100 a 1g.10gb instance is one seventh of the card, so the second file
above asks for roughly two sevenths of one GPU, not two GPUs.
That is deliberate (decision D5): the scheduler allocates discrete units it can bind to a VM, whether that unit is a PCI device or a carved MIG instance, so the count means the same thing in both cases even though the unit differs.
Where the rest of it lives
MIG (Multi-Instance GPU) is the NVIDIA feature behind gpu_profile: it splits
one datacenter card such as an A100 or H100 into up to seven hardware-isolated
instances, and a profile name like 1g.10gb reads as “1 of the card’s 7 compute
slices, 10GB of GPU memory”. The instances are carved by the operator ahead of
time, never on demand, so asking for a profile nobody carved is a scheduling
failure rather than a reconfiguration.
GPU workloads covers the
rest: how to read a profile name, when a slice beats a whole card, the exact
compile-time guardrails on these three fields, and how the scheduler picks a
host.
Scaffolding one
fuse init
writes a commented starter Fusefile to ./Fusefile (or a path passed via
-f/--file), and a .fuseignore next to it. Pass --force to
overwrite an existing file. The scaffold covers the common CPU case and omits
the GPU fields, add them by hand from the reference above.
The scaffold is intentionally maximal: its value is naming every field with a
comment, not being the shortest thing that runs. It is not a working file as
written, it declares a secret you must supply and an image that has to exist
on the host first. For the minimum that boots with a bare fuse up, see
Your first Fusefile.
Checking one
fuse validate
parses and compiles a Fusefile and reports every problem it finds, then exits 0
if the file is valid and 1 if it is not. It creates nothing and makes no
request, so it runs before fuse connect and in CI with no orchestrator
reachable. Pass -o json for a machine-readable report, or --quiet for the
exit code alone. See fuse validate.
Seeing what it compiles into
fuse compile
prints the resource spec, startup script, decoded manifest, exposed ports, and
required secret names a Fusefile produces, without creating anything and without
contacting the orchestrator. --format json or --format yaml prints the create
request body itself, which is what to diff when reviewing a Fusefile change.
Secret values are never accepted or printed, only the names. See
fuse compile.
Bringing it up
fuse up
reads a Fusefile (default ./Fusefile, or a positional path, or -f/--file),
compiles it, resolves required secrets from --secret key=value flags and/or a
--secrets-file, and creates an environment from the result, streaming
provisioning events until a terminal state, unless --no-wait is passed. If any
secret named in secrets is missing from the resolved set, up fails fast with the
list of missing names rather than creating a broken environment.
See fuse up, fuse init, and
fuse compile, and
fuse validate for the full flag reference.