The Case of the Phantom Files: Debugging ENOENT in a Next.js File Upload

Every developer has a story about an ENOENT error that just wouldn't quit. That's "Error NO ENTry" for the uninitiated, essentially meaning "file or directory not found." It's a simple error message, yet it often hides a labyrinth of complex interactions. I recently had my own wrestling match with ENOENT while working on our Axiom document upload feature, and the journey from baffling bug to triumphant fix was a masterclass in full-stack debugging.

The Mystery of the Missing Documents

Our goal was straightforward: allow users to upload ISO 27001 documents, which would then be processed by our backend for compliance checks. The frontend seemed to work fine – users could select files, the upload spinner would spin, and then... nothing. Or rather, the backend logs would scream ENOENT, complaining it couldn't find the very files it was supposed to be processing.

The symptoms were clear:

• User uploads a file (e.g., via a browser-initiated PUT request).
• Our processDocument function attempts to read this file from /tmp/nyxcore-uploads/....
• fs.readFile() throws ENOENT.

The logical conclusion? The file wasn't being written to disk before processDocument tried to read it. But why?

Unearthing the Culprits: A Tale of Two Bugs

It turns out, the problem wasn't one single issue, but a critical interplay of two distinct flaws:

Culprit #1: The Phantom API Route

Our application uses tRPC for type-safe API interactions. For file uploads, we were generating a presigned URL like /api/v1/uploads/{storageKey}. The intention was for the client to directly PUT the file content to this URL.

The catch? We had no actual HTTP route handler on the backend for this specific path!

This meant that when the client tried to PUT the file, it was silently hitting a 404. The browser's network tab might show a failed request, but our frontend's uploadMutation.onSuccess callback was still firing. Why? Because the fetch API doesn't throw an error on a 404, it just resolves with a response object containing the 404 status. If not explicitly checked, onSuccess proceeds as if everything is fine.

Culprit #2: The Premature Confirmation

This brings us to the second, equally critical, issue. In our client-side upload flow (src/app/(dashboard)/dashboard/projects/[id]/page.tsx), the uploadMutation.onSuccess callback contained a call to confirmMutation.mutate().

// Simplified, problematic client-side logic
uploadMutation.onSuccess(() => {
  // This fires even on a 404 if not explicitly checked!
  confirmMutation.mutate(); // Tells the backend to process the file
});

Because onSuccess was firing even when the PUT request failed (due to the missing route), confirmMutation.mutate() was being called before the file had any chance of being written to disk. Our backend would then immediately try to fs.readFile() a nonexistent file, leading to the infamous ENOENT.

It was a classic race condition, exacerbated by a silent API failure.

The Fix: Building the Missing Piece and Re-orchestrating the Dance

Once we understood the dual nature of the problem, the solutions became clear.

Fix 1: The Dedicated PUT Handler

We needed a robust HTTP PUT handler for /api/v1/uploads/[...path]. This handler's job would be to take the incoming file stream and write it securely to our designated temporary storage.

Here's a simplified look at what went into src/app/api/v1/uploads/[...path]/route.ts:

// src/app/api/v1/uploads/[...path]/route.ts
import { NextResponse } from 'next/server';
import { authenticateRequest } from '@/server/auth';
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { getStorageKey, getUploadDir } from '@/server/services/storage';
import { db } from '@/server/db';

export async function PUT(req: Request, { params }: { params: { path: string[] } }) {
  try {
    // 1. Authentication & Authorization
    const { user, tenantId } = authenticateRequest(req);
    // ... additional authorization logic for tenant/project ...

    const storageKey = params.path.join('/');
    // Validate storageKey against expected patterns (e.g., tenantId/projectId/filename)
    // Check if a ProjectDocument with this storageKey and status 'pending' exists in DB

    // 2. Security: Path Traversal Prevention & Safe Path Resolution
    const uploadDir = getUploadDir(tenantId); // e.g., /tmp/nyxcore-uploads/axiom/{tenantId}
    const targetFilePath = resolve(uploadDir, storageKey);

    // CRITICAL: Ensure targetFilePath does not escape uploadDir
    if (!targetFilePath.startsWith(uploadDir)) {
      return NextResponse.json({ error: 'Path traversal attempt detected' }, { status: 400 });
    }

    // 3. Content Type & Extension Filtering
    const contentType = req.headers.get('content-type');
    const filename = storageKey.split('/').pop();
    if (!filename || !['.pdf', '.docx', '.txt'].some(ext => filename.endsWith(ext))) {
      return NextResponse.json({ error: 'Unsupported file type' }, { status: 415 });
    }
    // Add X-Content-Type-Options: nosniff header for security
    const headers = new Headers();
    headers.set('X-Content-Type-Options', 'nosniff');

    // 4. Write the file to disk
    const arrayBuffer = await req.arrayBuffer(); // TODO: Stream for large files
    await writeFile(targetFilePath, Buffer.from(arrayBuffer));

    return NextResponse.json({ success: true, path: storageKey }, { status: 200, headers });

  } catch (error) {
    // ... error handling ...
    return NextResponse.json({ error: 'Upload failed' }, { status: 500 });
  }
}

This handler brought several crucial elements:

• Authentication & Tenant Isolation: Ensuring only authorized users can upload to their specific tenant's space.
• Path Traversal Prevention: Using path.resolve() and a subsequent check to ensure the resolved path doesn't escape our intended upload directory (/tmp/nyxcore-uploads/axiom/{tenantId}/). This is paramount for security.
• Extension Allowlist: Restricting uploads to specific, safe file types.
• Database Record Verification: Checking that a ProjectDocument with a status: "pending" already exists for this storageKey to prevent arbitrary file writes.
• X-Content-Type-Options: nosniff: A security header to prevent browsers from MIME-sniffing and potentially executing malicious files.
• Actual File Writing: Finally, taking the request body and writing it to the correct temporary location.

Fix 2: Re-orchestrating the Client-Side Confirmation

With the backend handler in place, the client-side logic needed adjustment. We moved confirmMutation.mutate() to only fire after a successful HTTP PUT response (i.e., status 200). We also added a continue statement on PUT failure to gracefully handle issues without attempting to process broken uploads.

// src/app/(dashboard)/dashboard/projects/[id]/page.tsx
// ...
uploadMutation.onSuccess(async (data, variables, context) => {
  // Check if the PUT request itself was successful (e.g., status 200)
  // Assuming 'data' now contains the response from our PUT handler
  if (data?.success) { // Or check 'data.status === 200' if not parsing JSON
    confirmMutation.mutate({
      projectId: variables.projectId,
      storageKey: variables.storageKey,
      fileSize: variables.file.size,
      mimeType: variables.file.type,
    });
  } else {
    console.error("File PUT failed, skipping confirmation.", data);
    // Optionally, show an error message to the user
  }
});

// Also, ensure the PUT request itself handles errors robustly
// For example, if using fetch:
// const response = await fetch(uploadUrl, { method: 'PUT', body: file });
// if (!response.ok) {
//   // Handle HTTP errors specifically
//   throw new Error(`Upload failed with status: ${response.status}`);
// }

This re-ordering ensured that processDocument would only be triggered after the file had been successfully written to disk, resolving the ENOENT errors.

Lessons Learned: From Pain to Progress

This session, though frustrating at times, offered invaluable insights:

1. Never Assume Your API Exists: Even with presigned URLs or generated paths, always verify that a corresponding backend route handler is in place and correctly configured. A silent 404 can be a devious bug.
2. Client-Side Race Conditions are Real: Be incredibly mindful of onSuccess callbacks in asynchronous operations. Ensure all preceding dependencies are truly met before triggering subsequent actions. Explicitly checking HTTP response status codes is critical.
3. Security is Not an Afterthought: File upload endpoints are prime attack vectors. Implementing robust security measures from the start—path traversal prevention, strict file type validation, and proper access control—is non-negotiable.
4. End-to-End Testing Catches These Glitches: A proper end-to-end test that simulates a user uploading a file and then verifies its processing would have caught this issue much earlier.

What's Next?

While the immediate ENOENT crisis is averted, the session also highlighted areas for future improvement:

1. Download Functionality: Implementing a GET handler for /api/v1/uploads/[...path] to allow LocalStorageAdapter.getDownloadUrl() to work.
2. Actual File Deletion: Currently, our LocalStorageAdapter.delete() is a no-op. We need to implement actual file cleanup to prevent disk bloat and ensure data hygiene.
3. Streaming Uploads: For very large files, req.arrayBuffer() can lead to out-of-memory (OOM) errors. Migrating to streaming body reads will make the system more robust.

Debugging is often a journey through the unknown, but each solved mystery strengthens our systems and our understanding. This ENOENT saga was a stark reminder of the intricate dance between frontend and backend, and the continuous need for vigilance in both design and implementation.

{"thingsDone":["Created a new PUT API route handler for browser-initiated file uploads (`/api/v1/uploads/[...path]`).","Implemented robust security measures in the PUT handler including authentication, tenant isolation, path traversal prevention, extension allowlisting, and DB record verification.","Fixed client-side upload flow to ensure file confirmation (`confirmMutation.mutate()`) only fires after a successful PUT response.","Conducted a security review and addressed critical issues related to file uploads."],"pains":["The tRPC presigned URL for uploads (`/api/v1/uploads/{storageKey}`) had no corresponding HTTP route handler, leading to silent 404s.","Client-side `uploadMutation.onSuccess` was firing prematurely, triggering file processing before the file was actually written to disk.","`fs.readFile()` on nonexistent files resulted in persistent `ENOENT` errors.","Difficulty in initial debugging due to silent HTTP failures masked by client-side optimism."],"successes":["Successfully resolved all Axiom document upload `ENOENT` errors.","User confirmed successful processing of ISO 27001 documents.","Improved security posture of the file upload mechanism.","Gained deeper insight into complex full-stack race conditions and silent API failures."],"techStack":["Next.js (App Router)","tRPC","Node.js (backend)","fs/promises (file system operations)","TypeScript","Database (for ProjectDocument status)"]}