File uploads and large file streaming

I've watched production systems grind to a halt because someone streamed a 2 GB video file through IFormFile into memory. File uploads and large file streaming is where the gap between a prototype and a platform becomes obvious. In this post, I'll show you how to handle uploads safely—from tiny profile pictures to massive data dumps—without crushing your server.

Why IFormFile isn't enough

IFormFile is convenient. You decorate a controller action, ASP.NET Core parses the multipart form, and you get a nice interface with .CopyToAsync(). But here's the catch: the entire file is buffered in memory first. For a 100 MB file on a server handling 20 concurrent uploads, you're looking at 2 GB of RAM gone in seconds.

On a project a few years back, we had file uploads crashing our e-commerce platform during peak hours. Not because the code was wrong—it worked fine in staging with 10 MB test files. But in production, customers were uploading high-resolution product photography and invoice PDFs. A single 500 MB upload would spike memory to the point where garbage collection couldn't keep up.

The solution wasn't to throw more RAM at the problem. It was to stop buffering altogether.

Stream directly to storage

Instead of reading the file into memory, you read from the request stream in small chunks and write directly to your storage backend—S3, Azure Blob Storage, or local disk. This keeps memory usage flat regardless of file size.

Here's how to configure your Kestrel server to allow larger request bodies:

csharp

var builder = WebApplicationBuilder.CreateBuilder(args);

builder.WebHost.ConfigureKestrel(o =>
{
    o.Limits.MaxRequestBodySize = 50 * 1024 * 1024; // 50 MB
});

var app = builder.Build();

Now, an upload controller that streams directly to S3:

csharp

[HttpPost("upload")]
public async Task<IActionResult> UploadFile()
{
    var request = HttpContext.Request;
    
    // Validate content type
    if (!request.ContentType.StartsWith("application/octet-stream"))
        return BadRequest("Invalid content type");
    
    var fileName = request.Headers["X-File-Name"].ToString();
    var fileSize = request.ContentLength ?? 0;
    
    // Validate size before streaming
    if (fileSize > 50 * 1024 * 1024)
        return BadRequest("File exceeds 50 MB limit");
    
    if (fileSize == 0)
        return BadRequest("Empty file");
    
    var s3Client = new AmazonS3Client();
    var bucketName = "my-uploads";
    var objectKey = $"uploads/{Guid.NewGuid()}_{fileName}";
    
    try
    {
        using (var stream = request.Body)
        {
            var putRequest = new PutObjectRequest
            {
                BucketName = bucketName,
                Key = objectKey,
                InputStream = stream,
                ContentLength = fileSize
            };
            
            await s3Client.PutObjectAsync(putRequest);
        }
        
        return Ok(new { url = $"https://{bucketName}.s3.amazonaws.com/{objectKey}" });
    }
    catch (Exception ex)
    {
        // Log and cleanup
        return StatusCode(500, "Upload failed");
    }
}

This approach never buffers the entire file. The stream reads from the client in 64 KB chunks (configurable) and writes to S3 simultaneously. Memory usage stays constant.

Presigned URLs: The better way

Streaming to storage is better than buffering, but there's a cleaner pattern: let the browser upload directly to S3 using a presigned URL. Your API only signs the request, then gets out of the way. This eliminates your server as a bottleneck entirely.

A presigned URL is a time-limited, cryptographically signed URL that grants temporary write access to a specific S3 object. The browser can upload directly without credentials:

csharp

[HttpPost("request-upload")]
public async Task<IActionResult> RequestUploadUrl([FromBody] UploadRequest req)
{
    // Validate on the API side
    if (req.FileSize > 5 * 1024 * 1024 * 1024) // 5 GB limit
        return BadRequest("File too large");
    
    // Validate extension
    var allowed = new[] { ".pdf", ".jpg", ".png", ".mp4" };
    var extension = Path.GetExtension(req.FileName).ToLower();
    if (!allowed.Contains(extension))
        return BadRequest("File type not allowed");
    
    var s3Client = new AmazonS3Client();
    var bucketName = "my-uploads";
    var objectKey = $"uploads/{Guid.NewGuid()}_{req.FileName}";
    
    // Create a presigned PUT URL valid for 15 minutes
    var presignedRequest = new GetPreSignedUrlRequest
    {
        BucketName = bucketName,
        Key = objectKey,
        Verb = HttpVerb.PUT,
        Expires = DateTime.UtcNow.AddMinutes(15),
        ContentType = req.ContentType
    };
    
    var presignedUrl = s3Client.GetPreSignedURL(presignedRequest);
    
    return Ok(new 
    { 
        url = presignedUrl,
        objectKey = objectKey
    });
}

public class UploadRequest
{
    public string FileName { get; set; }
    public string ContentType { get; set; }
    public long FileSize { get; set; }
}

On the frontend, the browser sends the presigned URL directly to S3:

csharp

// Pseudo-code for JavaScript
const { url, objectKey } = await fetch('/api/request-upload', {
    method: 'POST',
    body: JSON.stringify({ 
        fileName: file.name, 
        contentType: file.type,
        fileSize: file.size 
    })
}).then(r => r.json());

await fetch(url, {
    method: 'PUT',
    body: file,
    headers: { 'Content-Type': file.type }
});

I've used this pattern on a ride-hailing platform where drivers upload vehicle documents and trip recordings. It scaled beautifully—no request body size limits on our API, no memory spikes, and S3 handles the storage durability. Your server just validates and signs; S3 does the heavy lifting.

Validate everything

Whether you stream or use presigned URLs, validation is non-negotiable. Never trust the file name, Content-Type header, or file size reported by the client.

Size validation should happen before any upload starts. Content-Type validation should happen server-side, ideally with magic byte checking (content sniffing) rather than trusting the MIME type header:

csharp

public static class FileValidator
{
    private static readonly Dictionary<string, byte[]> MagicNumbers = new()
    {
        { "jpg", new byte[] { 0xFF, 0xD8, 0xFF } },
        { "png", new byte[] { 0x89, 0x50, 0x4E, 0x47 } },
        { "pdf", new byte[] { 0x25, 0x50, 0x44, 0x46 } },
        { "mp4", new byte[] { 0x00, 0x00, 0x00, 0x18, 0x66, 0x74, 0x79, 0x70 } }
    };
    
    public static bool IsValidFileType(Stream stream, string extension)
    {
        if (!MagicNumbers.TryGetValue(extension, out var expectedMagic))
            return false;
        
        var buffer = new byte[Math.Max(expectedMagic.Length, 8)];
        stream.Read(buffer, 0, buffer.Length);
        stream.Seek(0, SeekOrigin.Begin); // Reset for actual processing
        
        return buffer.Take(expectedMagic.Length)
            .SequenceEqual(expectedMagic);
    }
}

For the presigned URL approach, add validation middleware that checks the upload after it completes:

csharp

[HttpPost("confirm-upload")]
public async Task<IActionResult> ConfirmUpload([FromBody] ConfirmUploadRequest req)
{
    var s3Client = new AmazonS3Client();
    var bucketName = "my-uploads";
    
    // Download a small chunk to verify content type
    var getRequest = new GetObjectRequest
    {
        BucketName = bucketName,
        Key = req.ObjectKey,
        ByteRange = new ByteRange(0, 511) // First 512 bytes
    };
    
    using (var response = await s3Client.GetObjectAsync(getRequest))
    using (var reader = new StreamReader(response.ResponseStream))
    {
        var buffer = new byte[8];
        response.ResponseStream.Read(buffer, 0, 8);
        
        if (!IsValidMagicNumber(buffer, req.ExpectedType))
            return BadRequest("File content does not match declared type");
    }
    
    // Mark as confirmed in your database
    return Ok();
}

Common mistakes

Forgetting to reset MaxRequestBodySize. The default is 30 MB. Uploads over that silently fail with a 413 Payload Too Large error.
Buffering before validation. Don't read the entire stream into memory to validate it. Validate headers first, then stream.
Trusting file names. A user-supplied file name can contain path traversal sequences like ../../etc/passwd. Always generate a new name or sanitize aggressively.
No timeout on streaming. Set read timeouts on your stream operations. A slow client holding a connection open is a denial-of-service vector.
Ignoring presigned URL expiry. A leaked presigned URL is a security issue. Keep expiry windows short (5–15 minutes). Rotate credentials regularly.

Key takeaways

IFormFile buffers entire files into memory. For files over 10 MB or in high-concurrency scenarios, stream instead.
Stream directly to S3 or blob storage to keep server memory flat, even for multi-gigabyte uploads.
Use presigned URLs to let the browser upload directly to S3, removing your API server from the critical path entirely.
Validate file size, extension, and content type server-side. Use magic byte checking for content type, not just MIME headers.
Configure MaxRequestBodySize explicitly. The 30 MB default catches more people than you'd think.
Set read timeouts on streams. An idle connection is an attack surface.

File uploads and large file streaming: a practical guide