Blob uploads

A common requirement is to create an entity (say a Contract) and attach files to it. Over a JSON transport (JSON-RPC) the entity and its files cannot travel together — JSON cannot carry binary. So the file is uploaded separately, the upload returns a reference, and the entity-creation call carries that reference. This is the same shape S3 apps use: pre-upload, then reference.

The hard part is the orphan: if the user uploads a file but never finishes creating the entity (they cancel, or the browser closes), there is no backend signal. Elarion answers this the way S3 lifecycle rules do — with time plus an explicit promote:

A referenced upload is kept forever; an abandoned one is reclaimed. The TTL is the missing signal. This builds directly on blob storage and stays S3-free — the upload protocol lives only at the HTTP layer.

Package splitCopy Anchor Link

<ItemGroup>
  <PackageReference Include="Elarion.Blobs" Version="0.2.7" />
  <PackageReference Include="Elarion.Blobs.PostgreSql" Version="0.2.7" />
  <PackageReference Include="Elarion.Blobs.Tus" Version="0.2.7" />
</ItemGroup>

The lifecycleCopy Anchor Link

The lifecycle is a small capability over IBlobStore, provider-neutral and S3-free. BlobUploadRequest gains InitialState (defaults to Committed, so a plain SaveAsync stays permanent) and ExpiresAt; upload transports save with InitialState = Pending and an ExpiresAt. IBlobLifecycle adds two operations:

A Pending blob has two fates. Either an application commits it — the pre-upload-then-reference pattern below, which keeps it forever — or it is discarded: a consumed temp file is deleted explicitly (IBlobStore.DeleteAsync) and anything left over is reclaimed by the collector after its TTL (see temp-file flows). A blob is never both.

Register the PostgreSQL lifecycle and its background collector:

services.AddElarionPostgreSqlBlobLifecycle<AppDbContext>(); // store + lifecycle + GC sweeper
// (streaming reads clone the context's connection — no other wiring, ADR-0041)
// in OnModelCreating:
modelBuilder.UseElarionBlobStorage();

This adds state and expires_at columns plus a partial index over pending rows; create a migration (or EnsureCreated for throwaway databases) as for any model change.

Attaching a file to an entityCopy Anchor Link

public async ValueTask<Result<CreateContractResponse>> HandleAsync(CreateContract.Command request, CancellationToken ct) {
    db.Contracts.Add(contract);

    // Promote the pre-uploaded blob in the same transaction as the entity insert.
    if (!await blobLifecycle.CommitAsync(new BlobRef { Value = request.AttachmentBlobRef }, ct)) {
        return AppError.NotFound("attachment no longer available");
    }

    await db.SaveChangesAsync(ct); // commit + insert are atomic
    return new CreateContractResponse(contract.Id);
}

If the handler rolls back, the blob stays Pending and the collector reclaims it after Ttl + SafetyMargin. The collector deletes only rows still pending, so a commit that lands first wins the race and a never-committed upload is always reclaimed.

Upload transportsCopy Anchor Link

The lifecycle is transport-neutral. Two open transports produce Pending blobs over it; both stay S3-free. Pick by client needs.

tus — the resumable standardCopy Anchor Link

tus 1.0 is the open, resumable upload protocol built by the Uppy authors and the basis of the IETF Resumable Uploads draft. It is resumable, handles large files, and survives a browser close mid-upload, and is supported natively by Uppy (@uppy/tus) and tus-js-client. This is the recommended transport for new frontends.

services.AddElarionResumableBlobUploads();              // in-memory staging by default
app.MapElarionResumableBlobUploads().RequireAuthorization();

MapElarionResumableBlobUploads implements Creation, Core, Expiration, and Termination: OPTIONS advertises capabilities, POST creates an upload (baking the current user's id), PATCH/HEAD stream and resume, DELETE aborts. When an upload completes, the staged bytes are written as a Pending blob and its reference is returned in the Elarion-Blob-Ref response header (also available on HEAD) — the handle the client passes to entity creation.

Under the endpoints sits the protocol-neutral IStagedUploadStore seam: offset-guarded appends, an explicit idempotent completion (so a crash between the last chunk and completion self-heals on the next status probe), and deferred-length sessions — the shape of the IETF Resumable Uploads draft ("tus 2.0"), which will be a second adapter over the same seam. All policy (upload expiry, pending-blob TTL, completed- session retention) lives in ResumableBlobUploadOptions and reaches the store as data, so staging backends are protocol-free and interchangeable.

services.AddElarionResumableBlobUploads();
services.AddElarionPostgreSqlStagedUploads<AppDbContext>();
modelBuilder.UseElarionBlobStorage();  // blob tables, in OnModelCreating
modelBuilder.UseElarionStagedUploads(); // staging table, in OnModelCreating

Cross-origin frontends must expose the reference header via CORS: Access-Control-Expose-Headers: Upload-Offset, Location, Upload-Expires, Elarion-Blob-Ref. Behind a reverse proxy, enable UseForwardedHeaders so the Location header reflects the public scheme and host rather than the internal one.

Direct upload — minimal endpointCopy Anchor Link

For FilePond's process/revert and plain fetch/<form> clients, a single "accept bytes, return a reference" endpoint is the smallest path. It is not a new protocol.

services.AddElarionBlobUploads(o => o.MaxContentLength = 25 * 1024 * 1024);
app.MapElarionBlobUploads().RequireAuthorization();

POST (multipart or raw body) writes a Pending blob and returns its id as text/plain; DELETE /{id} cancels an owner's pending upload. Both enforce authentication, the configured size cap, and an optional content-type allow-list. Ownership is recorded as a dedicated BlobUploadRequest.OwnerId (surfaced as BlobMetadata.OwnerId) and compared exactly on cancel — not parsed from the stored name — so an id containing the naming separator cannot be forged, and a blob with no recorded owner is denied to everyone. The tus transport applies the same owner-exact, fail-closed check to its HEAD/PATCH/DELETE.

Client adaptersCopy Anchor Link

• Uppy — @uppy/tus against MapElarionResumableBlobUploads (recommended), or @uppy/aws-s3's getUploadParameters / @uppy/xhr-upload against the direct endpoint.
• FilePond — server.process → POST the direct endpoint (the response body is the file id); server.revert → DELETE /{id}. FilePond's chunked mode is tus-derived but not identical, so its resumable path needs a small client adapter.

Streaming downloads and temp-file flowsCopy Anchor Link

The staging area is also the framework's answer for large files in handler flows — the tier above the in-memory ElarionFile payload (which buffers and is the right default only up to a few megabytes). Handlers never carry big payloads; they carry pointers into the staging area, and the pending state doubles as temp-file semantics: what is never committed is garbage-collected after its TTL.

The read side is the owner-scoped streaming download endpoint:

app.MapElarionBlobDownloads().RequireAuthorization();   // GET {prefix}/{blobId}; requires AddElarionBlobUploads

GET /{blobId} streams the blob with its content type and leaf file name (Content-Disposition: attachment), applying the same exact-owner, fail-closed check as cancel — a missing, foreign-container, unowned, or someone else's blob is 404, so ownership is never leaked.

Import (large upload → processing handler):

1. The client uploads via tus (or the direct endpoint) into the staging area → gets a pending blob reference.
2. It calls the processing handler (any transport) with that reference as a plain string field.
3. The handler streams from IBlobStore.OpenReadAsync(...), processes, and disposes of the temp file.

A consumed import is the mirror image of attaching a file to an entity: you never keep it, so do not CommitAsync it — commit means "promote to permanent and clear the expiry," the opposite of what a throwaway import wants. There are two ways to reclaim it:

• Explicitly — delete it the moment you are done, so a large temp file is not left occupying storage for the whole TTL:

public async ValueTask<Result<Unit>> HandleAsync(ImportRows.Command command, CancellationToken ct) {
    var blobRef = new BlobRef { Value = command.BlobRef };
    await using var download = await blobs.OpenReadAsync(blobRef, ct);
    if (download is null) {
        return AppError.NotFound("upload expired");
    }

    await importer.IngestAsync(download.Content, ct);   // stream and process
    await blobs.DeleteAsync(blobRef, ct);               // reclaim now — no waiting for the GC
    return Unit.Value;
}

• Implicitly — do nothing, and the blob garbage collector reclaims the pending blob once its ExpiresAt passes. This is also the safety net if the process crashes before the explicit delete.

Note the handler works with the pending blob the upload produced, never the staging session: by the time it runs, the resumable upload has already been sealed into that blob (the session is cleaned up separately by the staged-upload collector, and must outlive the client's completion probe). "Removing it from staging" therefore means deleting the pending blob, exactly as above.

Export (handler-produced artifact → streaming download):

public async ValueTask<Result<ExportPointer>> HandleAsync(Command command, CancellationToken ct) {
    await using var artifact = await _exporter.BuildAsync(command.List, ct);   // a stream, never buffered
    var blobRef = await _blobs.SaveAsync(new BlobUploadRequest {
        Container = "uploads",                       // the download endpoint's configured container
        Name = $"{_currentUser.UserId}/{Guid.CreateVersion7():N}/{command.List}.xlsx",
        ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
        InitialState = BlobLifecycleState.Pending,   // temp: GC reclaims it if never downloaded/committed
        ExpiresAt = _timeProvider.GetUtcNow() + TimeSpan.FromMinutes(30),
        OwnerId = _currentUser.UserId,               // only the requesting user may download it
    }, artifact, ct);

    return new ExportPointer(blobRef.Value);         // the client GETs {prefix}/{id} to stream it
}

The response DTO carries only the blob id — small, cacheable, schema-friendly — and the actual bytes move once, streamed, over the download endpoint.

Streaming a ZIP archive into the response bodyCopy Anchor Link

"Download all of these as one file" is the one export shape the single-blob download endpoint cannot serve: the archive is assembled per request from several stored blobs and never exists as a blob of its own. Build it in a host endpoint that streams straight into HttpContext.Response.Body, so no intermediate copy is buffered in memory or staged in the database — the handler tier stays out of it, because neither ElarionFile (buffered, small-file tier) nor a Result<T> DTO can express "write bytes into the live response".

On .NET 10 that endpoint needs one deliberate opt-out, and it is worth understanding before you copy the recipe. System.IO.Compression gained async APIs (ZipArchive.CreateAsync, ZipArchiveEntry.OpenAsync, ZipArchive.DisposeAsync), but the entry stream returned by ZipArchiveEntry.OpenAsync does not override DisposeAsync. await using therefore falls through to the synchronous Dispose(), which writes the entry's data-descriptor trailer with a synchronous write on the response body. Under Kestrel — and under TestServer, which is how this usually surfaces first — AllowSynchronousIO is false by default, so that write throws InvalidOperationException: Synchronous operations are disallowed. There is no way to avoid it from the outside: CompressionLevel.NoCompression still writes the trailer, and there is no async flush that settles the entry before disposal.

The failure mode is nastier than a 500. By the time the first entry is closed, the 200 and the Content-Type: application/zip headers have already been flushed, so the exception cannot be turned into an error response — the client sees a application/zip download that simply never finishes.

The recipe: keep every genuinely async API, and widen the synchronous-I/O permission to this endpoint only.

using System.IO.Compression;
using Elarion.Blobs;
using Microsoft.AspNetCore.Http.Features;

app.MapGet("/exports/{id:guid}/bundle", async (
    Guid id,
    HttpContext http,
    IBlobStore blobs,
    IExportParts parts,
    CancellationToken ct) => {
    var members = await parts.ListAsync(id, ct);
    if (members.Count == 0) {
        return Results.NotFound();
    }

    // Scoped to this one endpoint: ZipArchive closes an entry with a synchronous write (see above).
    // Do not set this globally — the rest of the host keeps Kestrel's async-only default.
    var bodyControl = http.Features.Get<IHttpBodyControlFeature>();
    if (bodyControl is not null) {
        bodyControl.AllowSynchronousIO = true;
    }

    http.Response.ContentType = "application/zip";
    http.Response.Headers.ContentDisposition = $"attachment; filename=\"bundle-{id}.zip\"";

    var archive = await ZipArchive.CreateAsync(
        http.Response.Body, ZipArchiveMode.Create, leaveOpen: true, entryNameEncoding: null, ct);
    try {
        foreach (var member in members) {
            await using var download = await blobs.OpenReadAsync(new BlobRef { Value = member.BlobId }, ct);
            if (download is null) {
                continue;   // expired or already reclaimed — skip, do not fail the whole archive
            }

            var entry = archive.CreateEntry(member.FileName, CompressionLevel.NoCompression);
            await using var entryStream = await entry.OpenAsync(ct);
            await download.Content.CopyToAsync(entryStream, ct);
        }
    }
    catch (Exception) {
        // Headers are already on the wire, so a completed response would hand the client a
        // truncated archive that looks whole. Kill the connection instead — and deliberately leave
        // `archive` undisposed, so no central directory is written over a broken stream.
        http.Abort();
        throw;
    }

    await archive.DisposeAsync();   // writes the central directory
    return Results.Empty;
});

Three things carry the design:

• Blob bytes stay async and unbuffered. Each member is opened with OpenReadAsync and copied with CopyToAsync, so a large bundle never materializes — only the zip bookkeeping is synchronous.
• AllowSynchronousIO is per-request, not per-host. IHttpBodyControlFeature is resolved from HttpContext.Features and flipped on the way in, so the permission dies with the request. Setting KestrelServerOptions.AllowSynchronousIO = true globally to fix one endpoint gives every other endpoint a thread-blocking write path.
• A mid-stream failure aborts. HttpContext.Abort() is the only honest signal once the status line is gone; disposing the archive on the way out would append a valid-looking central directory to a partial archive.

Not S3 on the wireCopy Anchor Link

Elarion deliberately does not implement the S3 wire protocol (SigV4, aws-chunked streaming signatures, multipart XML). That is the interop-fragile, expensive part of S3, and upload widgets never need it — Uppy treats every upload URL as opaque. The S3 wire protocol is only required for real AWS SDK / CLI interoperability, which is a non-goal. The abstraction stays S3-free — Elarion.Blobs.Azure is exactly such a direct-to-storage backend behind the neutral seams, and a presigned-URL capability could be added the same way without changing the lifecycle.