Wire Protocol

Scope

pushduck's TypeScript packages are one implementation of a small HTTP contract. This page specifies that contract so any language can implement either half.

The key words MUST, MUST NOT, SHOULD and MAY are used as in RFC 2119.

Why there is a protocol at all

File bytes never pass through the upload server. The client asks for a presigned URL, PUTs directly to storage, and tells the server it finished:

Client ──1. presign──► Server ──► Storage (signs)
Client ──2. PUT ─────────────────► Storage (bytes)
Client ──3. complete─► Server        (hooks, DB writes)

The server is therefore a small, stateless signing and bookkeeping service — which is exactly why it is portable to Go, Python, or anything else.

Transport

• All requests MUST use HTTPS in production.
• Request and response bodies MUST be UTF-8 JSON with Content-Type: application/json, except errors (see Errors).
• A server MUST accept a request body of at least 100 KB on presign. It MAY reject larger bodies with 413.

Endpoint and dispatch

A server exposes one endpoint. Both the route and the operation are carried as query parameters:

POST  {endpoint}?route={routeName}&action=presign
POST  {endpoint}?route={routeName}&action=complete
GET   {endpoint}

POST  {endpoint}?route={routeName}&action=multipart-init
POST  {endpoint}?route={routeName}&action=multipart-sign
POST  {endpoint}?route={routeName}&action=multipart-complete
POST  {endpoint}?route={routeName}&action=multipart-abort
POST  {endpoint}?route={routeName}&action=multipart-parts

• route MUST be a route name defined by the server. A server MUST answer 404 for an unknown route.
• action MUST be one of the values above. When absent, a server MUST default to presign. An unrecognised action MUST answer 400.
• Route names SHOULD be percent-encoded by clients.

A conforming server MUST implement presign, complete and introspection. The five multipart-* actions are OPTIONAL: a server that omits them MUST answer 400 for each, which a client surfaces as an ordinary failure rather than misreading as success. See Multipart.

1. Presign

Asks the server to validate a batch and mint upload URLs.

Request — POST {endpoint}?route={routeName}&action=presign

{
  "files": [
    { "name": "photo.jpg", "size": 1048576, "type": "image/jpeg" }
  ],
  "metadata": { "albumId": "summer-2026" }   // optional, UNTRUSTED
}

Response — 200

{
  "success": true,
  "results": [
    {
      "success": true,
      "file": { "name": "photo.jpg", "size": 1048576, "type": "image/jpeg" },
      "presignedUrl": "https://bucket.s3.amazonaws.com/…?X-Amz-Signature=…",
      "key": "uploads/2026/photo.jpg",
      "requiredHeaders": { "Content-Type": "image/jpeg" },
      "metadata": { "userId": "u_1" },
      "completionToken": "eyJrZXkiOiJ1cGxvYWRzLzIwMjYvcGhvdG8uanBnIiwicm91dGUiOiJpbWFnZVVwbG9hZCJ9.…"
    }
  ]
}

results MUST be positionally aligned with the request's files.

Per-file versus whole-request failure

This distinction is normative and easy to get wrong:

• A failure describing one file (too large, wrong type, failed validation) MUST be reported as results[i].success = false inside a 200. One rejected file MUST NOT fail the others.
• A failure describing the request (unauthenticated, forbidden, over quota, unknown route, malformed body) MUST abort the batch and be returned as an error response with the appropriate status.

Burying a 401 inside a 200 makes it invisible to every status check, proxy, retry policy and alert.

2. Upload

The client PUTs the bytes directly to presignedUrl.

• The client MUST send every header in requiredHeaders, unmodified.
• When requiredHeaders is absent, the client SHOULD send Content-Type: {file.type}.
• The client MUST NOT send credentials; the URL is already signed.
• This request does not reach the pushduck server.

3. Complete

Tells the server which objects landed, so it can run post-processing.

Request — POST {endpoint}?route={routeName}&action=complete

{
  "completions": [
    {
      "key": "uploads/2026/photo.jpg",
      "file": { "name": "photo.jpg", "size": 1048576, "type": "image/jpeg" },
      "metadata": { "userId": "u_1" },
      "completionToken": "eyJrZXkiOiJ1cGxvYWRzLzIwMjYvcGhvdG8uanBnIiwicm91dGUiOiJpbWFnZVVwbG9hZCJ9.…"
    }
  ]
}

Response — 200

{
  "success": true,
  "results": [
    {
      "success": true,
      "key": "uploads/2026/photo.jpg",
      "url": "https://cdn.example.com/uploads/2026/photo.jpg",
      "presignedUrl": "https://…"        // optional, time-limited read URL
    }
  ]
}

A client SHOULD treat a failed complete as non-fatal: the bytes are already in storage, and reporting the upload as failed would be inaccurate.

Completion tokens (OPTIONAL)

Middleware authenticates the caller. It does not establish that the caller is finishing an upload they themselves started, because key travels in the request body — so an authenticated user can complete against a key belonging to someone else. Default keys are frequently the sanitised filename, which makes guessing one straightforward.

A server MAY return a completionToken from presign, binding the issued key to its route under a signature the client cannot forge. Where it does:

• A client SHOULD echo the token back in the matching completion entry.
• A server MUST verify a token that is present, and MUST answer 403 when the token's key or route does not match the completion.
• A server MUST accept a completion carrying no token, unless the route is configured to require one. Rejecting by default would break every client older than the token, including — during a rolling deploy — the previous build of the same application.

The token is opaque. Its contents, signing algorithm and secret are entirely the server's business; a client MUST treat it as a string to be echoed.

Multipart (OPTIONAL)

A single PUT cannot carry more than 5 GB, and on a mobile connection a failure at 90% discards everything transferred. Multipart splits one object into parts that upload independently, in parallel, each retried on its own.

A server MAY implement it. A server that does not MUST answer 400 to each multipart-* action, which a client surfaces as an ordinary failure. A client MUST NOT assume support: it discovers it by attempting init, and a 400 means fall back to a single PUT or fail the file honestly.

Every multipart action MUST be authorised exactly as presign is. The route's middleware runs on each call, so a revoked user cannot finish an upload they started.

Sessions

init returns an opaque session identifying the upload. Later calls present the session rather than a key and provider upload id.

4a. multipart-init

// request
{
  "file": { "name": "video.mp4", "size": 524288000, "type": "video/mp4" },
  "metadata": { "albumId": "summer-2026" },   // optional, UNTRUSTED
  "partSize": 5242880                          // optional, a preference
}

// response 200
{
  "success": true,
  "session": "…",                 // opaque
  "key": "uploads/2026/video.mp4",
  "partSize": 5242880,            // authoritative
  "metadata": { "userId": "u_1" }
}

The server's partSize is authoritative; a client MUST use it to compute part boundaries and MUST NOT use its own preference. Route validation and middleware run here, so a multipart upload cannot bypass a route's constraints.

4b. multipart-sign

// request
{ "session": "…", "partNumbers": [1, 2, 3] }

// response 200 — a bare array
[
  { "partNumber": 1, "url": "https://…", "size": 5242880 }
]

A server MUST reject a part number outside the plan implied by the session's total size and part size, because signing one would authorise a write past the end of the object.

4c. multipart-complete

// request
{
  "session": "…",
  "parts": [ { "partNumber": 1, "etag": "\"abc…\"" } ],
  "file": { "name": "video.mp4", "size": 524288000, "type": "video/mp4" },
  "metadata": { }
}

// response 200
{ "success": true, "key": "uploads/2026/video.mp4", "url": "https://…" }

• A client MUST send each part's etag exactly as the storage provider returned it, quotes included. Stripping them is rejected as InvalidPart after every byte has already been transferred.
• A server MUST send parts to the provider in ascending partNumber order; providers reject an unordered list.

4d. multipart-abort

// request
{ "session": "…" }

// response 200
{ "success": true }

Discards the session and its parts. Abandoned parts are billed until removed and do not appear in a normal object listing, so a client SHOULD abort on any permanent failure. A server MUST treat aborting an already-absent upload as success, since that is the desired end state.

4e. multipart-parts

// request
{ "session": "…" }

// response 200
{ "success": true, "parts": [ { "partNumber": 1, "etag": "\"abc…\"" } ] }

Reports which parts the provider actually holds, which is what makes resume possible: the client's own record is a hint, and the provider is the authority.

A server MUST follow provider pagination to completion. A truncated listing is indistinguishable from a complete one, and stopping early makes a resuming client re-upload parts the provider already has.

Part sizing

Constraints are the intersection of what every supported provider accepts:

Every part except the last MUST be the same size. That is stricter than S3 requires, and deliberate: Cloudflare R2 rejects an upload whose non-final parts differ in size, so uniform parts are the only sizing that works everywhere.

A server MUST raise the part size when the requested one would exceed the 10,000-part cap. At the 5 MiB floor that cap is reached at roughly 48.8 GiB.

Introspection

Request — GET {endpoint} · Response — 200

{
  "success": true,
  "protocolVersion": 1,
  "routes": [{ "name": "imageUpload", "type": "s3-upload" }]
}

Servers MUST implement this. It is how a client, conformance runner, or synthetic check discovers routes and negotiates version.

Advertising optional features

A server MAY include a features array naming the optional parts of the protocol it implements:

{
  "success": true,
  "protocolVersion": 1,
  "routes": [ { "name": "imageUpload", "type": "s3-upload" } ],
  "features": ["multipart"]            // optional
}

Without it, a client can only discover multipart support by attempting multipart-init and interpreting a 400 — which is indistinguishable from a malformed request. A client MUST tolerate the field being absent and SHOULD treat absence as "unknown" rather than "unsupported", falling back to attempting the operation.

Defined feature names: multipart.

Errors

Errors MUST be RFC 9457 problem documents with Content-Type: application/problem+json, and the HTTP status MUST match the document's status.

{
  "type": "https://pushduck.org/errors/file-too-large",
  "title": "File exceeds the maximum size",
  "status": 413,
  "detail": "photo.jpg is 9.0 MB; the limit is 5.0 MB",
  "instance": "/api/upload?route=imageUpload&action=presign",
  "code": "FILE_TOO_LARGE",
  "retryable": false,
  "meta": { "limit": 5242880, "actual": 9437184 },
  "error": "photo.jpg is 9.0 MB; the limit is 5.0 MB"
}

code, retryable, meta and error are RFC 9457 extension members. Clients MUST branch on code, never on title or detail. error mirrors detail for clients predating this specification and MAY be omitted by a new implementation.

Codes

STORAGE_ACCESS_DENIED is 502, not 403: the caller is authorised — the server's own storage credentials are not. A 403 would wrongly tell the client to re-authenticate.

An implementation MAY add codes. A client MUST treat an unrecognised code as INTERNAL_ERROR and fall back to the HTTP status.

Redaction

• For 4xx, a server SHOULD return detail and meta — they describe the caller's own request.
• For 5xx, a server SHOULD replace detail with title and omit meta unless explicitly configured otherwise; they may describe server internals.

Response headers

These exist so infrastructure can distinguish operations. Query strings are legitimately dropped or ignored by CDNs, gateways and APM tools, which would otherwise collapse every upload request into one indistinguishable line.

Versioning

• PROTOCOL_VERSION is bumped only for a breaking change — a new required field, a changed URL shape, a removed operation.
• Additive changes (a new optional field, a new action) MUST NOT bump it. Clients MUST ignore unknown fields.
• A server SHOULD accept older protocol versions indefinitely. Deployments are not atomic: a browser holding a cached bundle will speak the old version to a new server for as long as its cache lives.

Added since version 1 shipped

All additive, so none bumped the version. An implementation may omit any of them and still conform; a client MUST tolerate their absence.

Implementing a server in another language

You need three handlers, and a SigV4 presigner from your ecosystem (aws-sdk-go-v2's presign client, boto3's generate_presigned_url).

// Go — net/http, no dependencies beyond the AWS SDK
func handler(w http.ResponseWriter, r *http.Request) {
    route  := r.URL.Query().Get("route")
    action := r.URL.Query().Get("action")
    if action == "" { action = "presign" }
    // dispatch on (r.Method, action)
}

# Python — FastAPI
@app.post("/api/upload")
async def upload(route: str, action: str = "presign", body: dict = Body(...)):
    ...

Query-string dispatch is deliberate here: r.URL.Query() and request.query_params are stdlib in both languages and behave identically, whereas path parameters differ per web framework and would force the server to know its own mount prefix.

Conformance

An implementation conforms when it:

1. Implements presign, complete and introspection at one endpoint
2. Reports protocolVersion and the X-Pushduck-Protocol header
3. Returns RFC 9457 problem documents with matching HTTP statuses
4. Distinguishes per-file from whole-request failures as specified
5. Aligns results positionally with the request's files
6. Authorises complete with the route's middleware
7. Answers 400 for any optional action it does not implement
8. Does not depend on its own mount path

Checking it

That list is prose, and prose is not enough. Point 4 is the rule most often implemented backwards — a file failing validation is a 200 with results[].success: false, not a 4xx — and the author of the reference implementation read it wrong while writing tests for their own server.

So the rules are also an executable suite. It is language-neutral JSON plus a runner, and it tests a server over HTTP without caring what it is written in:

pnpm conformance --url http://localhost:8080/api/upload

The suite is in conformance/, along with the fixed route surface an implementation must expose for the fixtures to assert anything about validation. Each fixture names the section of this page it comes from, so a failure points at the rule rather than only at the assertion.

A reference server is included for the ambiguous cases:

pnpm conformance:serve   # then diff your responses against it

Two implementations pass it today: the TypeScript server in packages/pushduck, and the Go server in packages/pushduck-go. Their SigV4 signatures are byte-identical for the same inputs, which is what allows a client to presign against one and complete against the other.