← nyxcore-systems
5 min read

Unmasking the Phantom File: Our Journey to Taming `ENOENT` in File Uploads

We faced a baffling `ENOENT` error in our document upload feature. This post details how we debugged a silent 404, a client-server race condition, and built a robust, secure upload pipeline.

Next.jsFileUploadsDebuggingENOENTAPI DevelopmentTypeScriptSecuritytRPC

Every developer knows the dreaded ENOENT error – "Error: No such file or directory." It's a classic, but when it pops up in a critical feature like document uploads, it can be particularly frustrating. Recently, we encountered this very nemesis while working on the Axiom document upload feature, where files seemed to vanish into the ether right after being "uploaded."

This post chronicles our debugging adventure, the surprising culprits we uncovered, and the robust solution we implemented to ensure our ISO 27001 documents could finally find their way to disk.

The Mystery of the Missing Files

Our goal was simple: allow users to upload important documents, have them processed, and stored. The front-end indicated success, but when the backend tried to read the files for processing, it consistently threw ENOENT. It was a classic "it worked on my machine (the client's browser)" vs. "it didn't work on the server" scenario.

The initial setup involved a tRPC endpoint (axiom.upload) which returned a presigned URL like /api/v1/uploads/{storageKey}. The client was then supposed to PUT the file directly to this URL. Our backend processDocument function would then pick up the file from a temporary storage location and do its magic.

The Silent Killer: A Missing Route

Our first major discovery was both simple and infuriating: there was no actual route handler for that /api/v1/uploads/{storageKey} URL.

Yes, you read that right. The tRPC call successfully returned the URL, but the server didn't have a handler to receive the file at that URL. This meant the browser's PUT request was silently failing with a 404, or perhaps a 405 (Method Not Allowed), without the client's onSuccess callback ever truly knowing the file hadn't been written.

The Race Condition: Firing Too Soon

Compounding the problem was a classic asynchronous pitfall. On the client side, our uploadMutation.onSuccess callback was prematurely calling confirmMutation.mutate(). This confirmMutation would tell the backend that the file was ready for processing.

The sequence of events was:

  1. User initiates upload.
  2. uploadMutation starts (but doesn't complete the PUT request).
  3. uploadMutation.onSuccess fires (because the initial tRPC call for the presigned URL succeeded, not the actual file upload).
  4. confirmMutation tells the server the file is ready.
  5. Server tries to fs.readFile() the file.
  6. BOOM! ENOENT. The file hadn't even been sent or written to disk yet because the PUT request was still pending (or silently failing).

It was a perfect storm of a missing route handler and a client-side race condition. No wonder the files were phantom!

Building a Robust Upload Pipeline

Solving this required a two-pronged approach: creating the missing backend handler and re-sequencing the client-side logic.

1. The Backend: A Dedicated Upload Handler

We created a new Next.js API route: src/app/api/v1/uploads/[...path]/route.ts. This PUT handler is now responsible for receiving the file and writing it to our temporary storage.

Key features of this new handler:

  • Authentication & Authorization: Using authenticateRequest(), we ensure only authorized users can upload.
  • Tenant Isolation: Files are stored in tenant-specific directories (/tmp/nyxcore-uploads/axiom/{tenantId}/...) to prevent cross-tenant data leakage.
  • Path Validation & Security: path.resolve() with careful boundary checks prevents path traversal attacks, ensuring files are written only within designated upload directories.
  • Extension Allowlist: Only approved file types (e.g., PDFs, JPEGs) can be uploaded, mitigating risks associated with arbitrary file uploads.
  • Database Record Verification: Before writing, we verify that a ProjectDocument record with a status: "pending" exists in our database, linking the upload to a legitimate project and preventing unsolicited file writes.
  • Security Headers: We added X-Content-Type-Options: nosniff to prevent browsers from MIME-sniffing and potentially misinterpreting file types.

Files are now safely written to paths like /tmp/nyxcore-uploads/axiom/{tenantId}/{projectId}/{timestamp}-{filename}.

2. The Frontend: Correcting the Flow

On the client side, in src/app/(dashboard)/dashboard/projects/[id]/page.tsx, we made a critical adjustment:

  • Delayed Confirmation: The call to confirmMutation.mutate() was moved from uploadMutation.onSuccess to after the successful PUT response from our new /api/v1/uploads/[...path] handler. This ensures the server is only notified once the file has genuinely landed on disk.
  • Robust Error Handling: We added a continue statement on PUT failures, allowing the client to skip broken uploads and prevent subsequent confirmMutation calls for files that were never successfully sent.

Lessons Learned & Security Spotlight

This debugging session offered several valuable insights:

  • Never Assume: Just because a tRPC call returns a URL, don't assume the corresponding HTTP handler exists or works as expected. Always verify the entire request lifecycle.
  • Silent Failures are Insidious: A 404 on a PUT request can be easily missed if not explicitly handled or monitored. Robust logging and client-side error handling are crucial.
  • Race Conditions are Everywhere: Asynchronous operations are a prime breeding ground for race conditions. Carefully sequence your client-server interactions, especially when multiple steps depend on the completion of previous ones.
  • Security First: Even in a debugging sprint, security cannot be an afterthought. Integrating path validation, tenant isolation, and extension allowlists from the start prevents major vulnerabilities down the line. We explicitly conducted a security review to address potential path traversal and unrestricted upload issues.

What's Next?

Our current fix addresses the immediate ENOENT problem, but a robust file storage system is an evolving beast. Here are our immediate next steps:

  1. Download Functionality: Implement a GET handler for /api/v1/uploads/[...path] so LocalStorageAdapter.getDownloadUrl() can serve the uploaded files back to users.
  2. Actual File Deletion: Our LocalStorageAdapter.delete() is currently a no-op. We need to implement proper file cleanup to manage storage space and data lifecycle.
  3. Streaming for Large Uploads: For very large files, req.arrayBuffer() can lead to Out-Of-Memory (OOM) errors. We'll explore streaming body reads to handle uploads more efficiently.

This journey from baffling ENOENT to a fully functional and secure document upload pipeline was a valuable reminder of the complexities of web development. By carefully dissecting the problem, understanding the full request lifecycle, and prioritizing security, we've built a more resilient system for Axiom.