The Case of the Missing Uploads: Conquering ENOENT in a Next.js App

Every developer has faced the dreaded ENOENT error – "Error: No such file or directory." It's often straightforward: a file path is wrong, or a file genuinely doesn't exist. But what happens when you're sure the file should be there, and yet, ENOENT persists? This was the puzzle we recently untangled in our application, specifically impacting our Axiom document upload feature for critical ISO 27001 compliance documents.

It was a classic case of files vanishing into thin air between the browser and the server, leading to processing failures. Today, I'll walk you through the journey of diagnosing and fixing this elusive bug, highlighting the technical decisions and crucial security measures we implemented along the way.

The Mystery Unfolds: When Files Just Don't Exist (But Should)

Our goal was simple: allow users to upload documents, have them processed, and update their status. The workflow involved a client-side upload initiated via tRPC, which would receive a presigned URL, and then the client would PUT the file to that URL. Following a successful PUT, a confirmation mutation would trigger the backend processing.

The problem? Users were uploading documents, the UI reported success, but the backend kept failing with ENOENT when trying to read the uploaded files. This was baffling, especially since our other REST API-based upload path (/api/v1/rag/ingest) was working perfectly.

Here's what was happening under the hood:

1. The Silent 404: Our tRPC axiom.upload did return a presigned URL like /api/v1/uploads/{storageKey}. However, a critical piece was missing: there was no actual route handler for that URL. The client was dutifully sending a PUT request, but it was hitting a silent 404 or an unhandled route, meaning the file data was never being written to disk.
2. The Premature Celebration: Compounding the issue, our client-side code was a bit too optimistic. The confirmMutation.mutate() call – which tells the backend to start processing the document – was firing within the onSuccess callback of the upload mutation. This meant that as soon as the browser thought it had initiated the upload (even if it silently failed), it would tell the backend to process a file that hadn't even started uploading, let alone been written to disk.
3. The Inevitable ENOENT: When the backend's processDocument function tried to fs.readFile() on a file that was never written, ENOENT was the only logical outcome.

It was a classic race condition combined with a missing API endpoint – a recipe for confusion and frustration.

The Solution: Building the Missing Pieces and Reordering the Flow

To resolve this, we tackled both the missing backend API and the flawed client-side logic.

1. Constructing the Upload API (src/app/api/v1/uploads/[...path]/route.ts)

The first order of business was to create the PUT handler that was supposed to receive the file. We built this as a Next.js API Route, carefully considering security and robustness:

• Authentication & Tenant Isolation: Every request goes through authenticateRequest() to ensure only authorized users are uploading, and tenantId is used to segregate uploads, preventing cross-tenant data leakage.
• Path Traversal Protection: We used path.resolve() with strict boundary checks to prevent malicious users from uploading files outside their designated storage directory (e.g., trying to write to /etc/passwd). This is a non-negotiable security measure for any file upload feature.
• Extension Allowlist: Only specific file extensions (e.g., .pdf, .docx) are permitted. This helps mitigate risks associated with executable files or other potentially harmful types.
• Database Record Verification: Before writing the file, we verify that a ProjectDocument record with a status: "pending" exists in the database for the given storageKey. This ensures that only uploads for expected documents are processed, preventing arbitrary file writes.
• Security Headers: We added X-Content-Type-Options: nosniff to prevent browsers from MIME-sniffing and potentially misinterpreting the content type, which can be an attack vector.
• File Storage: Files are now correctly written to a temporary directory structure like /tmp/nyxcore-uploads/axiom/{tenantId}/{projectId}/{timestamp}-{filename}.

This new backend route now correctly receives the file stream, writes it to disk, and signals success.

2. Refining the Client Upload Flow (src/app/(dashboard)/dashboard/projects/[id]/page.tsx)

With the backend ready, we adjusted the client-side logic to ensure the confirmMutation only fires after a successful file upload:

• Delayed Confirmation: We moved confirmMutation.mutate() out of the uploadMutation.onSuccess callback. Instead, it now only triggers after the PUT request to the presigned URL has successfully completed and returned a success response. This ensures the file is truly on disk before we ask the backend to process it.
• Robust Error Handling: We added a continue statement on PUT failure. This means if an individual file upload fails (e.g., network error, API returns an error), the client won't try to confirm that specific broken upload, and it can continue with other successful uploads in a batch.

Lessons Learned: The Takeaways

This debugging session reinforced several critical lessons:

• Don't Assume Success: Never assume an operation has completed successfully until you've received explicit confirmation. Our premature confirmMutation was a classic example of "celebrating too early."
• Comprehensive API Design: If you expose an endpoint, ensure it has a handler. Silent failures (like a 404) can be incredibly hard to debug.
• Security First for File Uploads: File upload features are prime targets for attackers. Always implement:
  • Strong authentication and authorization.
  • Path traversal protection.
  • Strict file type validation (allowlists, not blocklists).
  • Size limits.
  • Content-type sniffing prevention.
• Trace the Full Flow: When debugging, trace the entire data flow from the UI, through all API calls, to the final storage and processing steps. This helps pinpoint where the disconnect truly lies.

What's Next? Continuous Improvement

While the immediate ENOENT issue is resolved, and users are now successfully uploading their ISO 27001 documents (confirmed with commit baf22ab to origin/main!), there are always next steps for a robust system:

1. Download Functionality: Implement a GET handler for /api/v1/uploads/[...path] so LocalStorageAdapter.getDownloadUrl() can actually serve the uploaded files.
2. File Deletion: Our current LocalStorageAdapter.delete() is a no-op. We need to implement actual file deletion for proper cleanup and data hygiene.
3. Streaming Uploads: For very large files, req.arrayBuffer() can lead to out-of-memory (OOM) errors. We should consider migrating to streaming body reads to handle uploads more efficiently.

This journey from a perplexing ENOENT to a fully functional and secure document upload system underscores the importance of meticulous API design, careful client-server interaction, and an unwavering focus on security. It's these kinds of challenges that make development both frustrating and incredibly rewarding!

---