Unmasking the Silent 404: How a Missing API Route Broke Our File Uploads

Every developer knows the unique blend of frustration and exhilaration that comes with chasing down a cryptic bug. Recently, our team encountered one such challenge: critical ISO 27001 compliance documents were consistently failing to upload to our Axiom platform, throwing a baffling ENOENT (Error No ENTity) error. This isn't just a minor glitch; it's a showstopper when you're dealing with essential compliance data.

This post chronicles our journey from a head-scratching ENOENT to a robust, secure file upload mechanism.

The Mystery of the Missing File

The symptom was clear: documents would appear to upload from the client's perspective, but when our backend tried to fs.readFile() them for processing, it would crash with ENOENT. It was as if the file was never written to disk. But how could that be? Our client-side upload mutation was reporting onSuccess!

Our existing upload flow for Axiom documents involved a few steps:

1. A tRPC call (axiom.upload) would initiate the upload process.
2. The server would respond with a presigned URL, typically pointing to an internal API endpoint like /api/v1/uploads/{storageKey}.
3. The client would then directly PUT the file content to this presigned URL.
4. Finally, after the client's PUT operation completed (or so we thought), a confirmUpload mutation would be triggered, signaling the backend to process the now-uploaded file.

This seemed straightforward enough. We even had a similar REST API path (/api/v1/rag/ingest) that handled file uploads perfectly, writing the file directly within its request handler. Why was the tRPC-initiated flow failing?

Unmasking the Silent 404

The "aha!" moment came when we dug deeper into the network requests. While the client-side onSuccess callback was firing, indicating the PUT request was sent, the server's response was a silent, unassuming 404 Not Found for the /api/v1/uploads/{storageKey} endpoint.

That's right. The presigned URL pointed to an endpoint that simply didn't exist!

Our mental model assumed that because the axiom.upload tRPC procedure generated the URL, a corresponding handler must exist. But in this asynchronous, multi-step flow, that critical piece was missing. The client was happily PUTting data into the void, getting a 404, and then immediately triggering confirmUpload because its uploadMutation.onSuccess handler fired as soon as the request completed, not necessarily when the file was successfully stored.

This explained everything:

• Silent 404: The client's PUT request was rejected, but the onSuccess callback was triggered regardless, leading to a false sense of security.
• Race Condition: Because onSuccess fired prematurely, the confirmUpload mutation was racing ahead, telling the backend to process a file that was never actually written.
• ENOENT: When processDocument tried to read the file, it genuinely wasn't there.

Building the Missing Piece: A Robust Upload Handler

With the root cause identified, the path forward became clear: we needed a dedicated, robust API route to handle these direct client PUT requests.

We introduced src/app/api/v1/uploads/[...path]/route.ts. This new Next.js API route is designed to be a secure and reliable entry point for file uploads. Here's a breakdown of its key features:

• Authentication & Authorization: Every request is authenticated using authenticateRequest(), and critically, we enforce tenant isolation. The tenantId extracted from the URL path must match the tenantId of the authenticated session, preventing cross-tenant data leakage.
• Path Traversal Prevention: We use path.resolve() to assert the boundary of the upload directory, mitigating potential path traversal attacks where malicious users try to write files outside the intended upload folder.
• Extension Allowlist: To prevent the upload of potentially harmful or unexpected file types, we enforce a strict allowlist of extensions (.md, .txt, .pdf, .ts, .js, .py, .json, .yaml, .yml, .toml, .html, .css).
• Database Verification: Before accepting the upload, we verify that a ProjectDocument with the matching storageKey already exists in a "pending" status in our database. This ensures that only expected and pre-registered uploads are processed.
• Security Headers: We added X-Content-Type-Options: nosniff to prevent browsers from trying to "guess" the content type, which can be a security risk.

// src/app/api/v1/uploads/[...path]/route.ts (Simplified for brevity)
import { authenticateRequest } from '@/lib/auth';
import { NextResponse } from 'next/server';
import path from 'node:path';
import { writeFile } from 'node:fs/promises';
import { db } from '@/lib/db';

const ALLOWED_EXTENSIONS = [
  '.md', '.txt', '.pdf', '.ts', '.js', '.py', '.json', '.yaml', '.yml', '.toml', '.html', '.css'
];

export async function PUT(req: Request, { params }: { params: { path: string[] } }) {
  const session = await authenticateRequest();
  if (!session) {
    return NextResponse.json({ message: 'Unauthorized' }, { status: 401 });
  }

  const [tenantId, projectId, ...restPath] = params.path;
  const filename = restPath.join('/');
  const storageKey = path.join(tenantId, projectId, filename); // Construct storage key from path

  if (tenantId !== session.tenantId) {
    return NextResponse.json({ message: 'Forbidden: Tenant ID mismatch' }, { status: 403 });
  }

  const document = await db.projectDocument.findUnique({
    where: { storageKey, status: 'pending' },
  });

  if (!document) {
    return NextResponse.json({ message: 'Document not found or not in pending state' }, { status: 404 });
  }

  const fileExtension = path.extname(filename).toLowerCase();
  if (!ALLOWED_EXTENSIONS.includes(fileExtension)) {
    return NextResponse.json({ message: 'File type not allowed' }, { status: 400 });
  }

  // Ensure file is written to a safe, controlled directory
  const uploadDir = path.resolve(process.env.UPLOAD_DIR || '/tmp/nyxcore-uploads/axiom');
  const targetPath = path.join(uploadDir, storageKey);

  // Prevent path traversal
  if (!targetPath.startsWith(uploadDir)) {
    return NextResponse.json({ message: 'Invalid path' }, { status: 400 });
  }

  try {
    const arrayBuffer = await req.arrayBuffer();
    await writeFile(targetPath, Buffer.from(arrayBuffer));
    return new NextResponse(null, { status: 200, headers: { 'X-Content-Type-Options': 'nosniff' } });
  } catch (error) {
    console.error('File write error:', error);
    return NextResponse.json({ message: 'Failed to write file' }, { status: 500 });
  }
}

Fixing the Client-Side Race

The server-side API route was only half the battle. We also adjusted the client-side upload flow in src/app/(dashboard)/dashboard/projects/[id]/page.tsx. We moved the confirmMutation.mutate() call from uploadMutation.onSuccess to after the successful PUT response. This ensures that the backend is only notified to process a document after the file has actually been written to disk.

We also added a continue statement on PUT failure, preventing any subsequent processing steps from firing if the upload itself failed, further hardening our upload mechanism.

Lessons Learned

This debugging session offered some valuable insights:

1. Verify All API Surface Areas: Never assume a presigned URL or an implicit API endpoint actually has a handler. Always verify that all parts of your API contract are explicitly implemented.
2. Don't Trust onSuccess Blindly in Asynchronous Flows: In multi-step asynchronous processes, onSuccess often means the request was sent, not necessarily that the operation completed successfully on the server. Always check server responses for actual success (e.g., a 2xx status code).
3. Robust Error Handling is Key: The silent 404 was particularly insidious. Better logging or explicit error handling on the client could have surfaced this much faster.
4. Security First in File Uploads: File upload endpoints are prime targets for attacks. Always implement strict authentication, authorization, path validation, and content type restrictions.

What's Next?

With the critical ENOENT error resolved and our ISO 27001 document uploads now working reliably, we've committed the fix (baf22ab) and are ready to push to main.

Our immediate next steps include:

• Pushing the fix to origin/main and re-uploading the ISO 27001 documents to verify successful processing end-to-end.
• Considering adding a GET handler for /api/v1/uploads/[...path] to enable LocalStorageAdapter.getDownloadUrl() functionality.
• Implementing actual file deletion in LocalStorageAdapter.delete() (it's currently a no-op).
• Investigating streaming body reads instead of req.arrayBuffer() to prevent out-of-memory errors with very large file uploads.

This journey from a perplexing ENOENT to a clear, robust solution highlights the importance of meticulous debugging, understanding your full API surface, and building security into every layer of your application. Happy coding!

{
  "thingsDone": [
    "Created src/app/api/v1/uploads/[...path]/route.ts (PUT handler for local file uploads)",
    "Implemented authentication and tenant isolation in the new PUT handler",
    "Added path traversal prevention and extension allowlist to upload handler",
    "Integrated