From Code to Content: nyxBook's Latest Evolution with GitHub & Zip Imports

Development sessions are often a whirlwind of focused energy, problem-solving, and the satisfying click of features falling into place. Our latest session for nyxBook was no exception, culminating in a significant leap forward in how users can bring their stories into the platform. We tackled two major enhancements: robust GitHub repository integration and convenient zip file uploads for content, all while giving our main navigation sidebar a sleek, consistent makeover.

By the end of the session, all features were implemented, thoroughly type-checked and linted, and even passed a rigorous code review. We're now primed for deployment, and I'm excited to share the journey with you.

The Quest for Seamless Content Ingestion

Our primary goal was to drastically improve the ease with which users could populate nyxBook with their creative projects. Manual entry is fine for small tweaks, but for importing entire book structures—characters, beats, chapters—we needed something more powerful.

GitHub Integration: Bringing Your Repos to Life

Imagine your story drafts living comfortably in a GitHub repository. Now, imagine pointing nyxBook directly at that repository and watching your content magically appear, structured and ready for editing. That's precisely what we built.

Our new src/server/services/nyxbook-github.ts service is the brain behind this operation. It intelligently checkGitHubForBook(), scanning your chosen repository for a conventional book/ directory, which indicates a structured nyxBook project. Once confirmed, importFromGitHub() springs into action, fetching the content directly via the GitHub API (no cloning needed, keeping things lean and secure!). This content is then fed into our shared parsing services, which meticulously break down chapters, characters, beats, and personas, transforming them into structured database records.

On the frontend, src/app/(dashboard)/dashboard/nyxbook/page.tsx now features a beautifully redesigned import section. It includes a searchable repository combobox with type-to-filter functionality, making it a breeze to find your projects. The system even auto-scans for the book/ directory as soon as you select a repository, providing instant feedback.

Zip Uploads: Drag, Drop, and Done

For those who prefer a local workflow or have content not hosted on GitHub, we introduced a highly requested zip upload feature. The src/app/api/v1/nyxbook/import/route.ts endpoint now handles multipart/form-data uploads, using JSZip to extract archives.

Security and robustness were paramount here:

• Auto-detection: Our findBookRoot() function intelligently detects the book's structure within the zip, even if it's nested.
• Path Traversal Protection: Critical measures are in place to prevent malicious path traversal attempts.
• Strict Limits: We enforce a 20MB file size limit and a 200-entry limit for extraction, safeguarding against denial-of-service attacks.
• .md-only Filter: Only markdown files are processed, ensuring content relevance and reducing potential risks.
• Cleanup: Temporary directories are diligently cleaned up after processing.

The UI also boasts a user-friendly drag-and-drop zone, making the upload process incredibly intuitive.

The Parsing Powerhouse and API Guardians

At the heart of both import methods lies our refined src/server/services/nyxbook-import.ts. We've modularized and exported key parser functions like parseCharacters, parseBeats, splitDualLayer, and countWords, promoting reusability and maintainability. A crucial security enhancement here was sanitizing metadata to prevent any accidental leakage of internal filesystem paths.

Our tRPC router (src/server/trpc/routers/nyxbook.ts) now includes checkGitHub and importGitHub procedures. These endpoints are fortified with robust regex validation for GitHub owner (max 39 characters, specific pattern) and repository names (max 100 characters, specific pattern), along with a branch name limit of 256 characters, ensuring that only valid inputs make it through.

A Fresh Coat of Paint: The Sidebar Redesign

While the import features were a heavy lift, we also took the opportunity to refine the overall user experience by redesigning the main application sidebar (src/components/layout/sidebar.tsx). The goal was to align its aesthetics and functionality with our "InPageSidebar" style, creating a more consistent and polished feel across the application.

This involved a series of subtle yet impactful CSS adjustments:

• Compactness: text-xs for items, making navigation feel snappier.
• Visual Flair: rounded links for a softer look.
• Clear Active State: bg-nyx-accent/10 provides a subtle yet distinct highlight for the active page.
• Subtle Grouping: text-[9px] group labels with /60 opacity enhance readability without being overbearing.
• Optimized Spacing: gap-2 for better visual separation.
• Responsive Widths: Intelligent w-12/w-48 widths for collapsed and expanded states.
• Icon Sizing: h-3.5 for icons, ensuring visual harmony.

These changes contribute to a cleaner, more modern, and ultimately more enjoyable navigation experience.

Fortifying the Walls: Security & Robustness

Security is a continuous process, and our code review brought several key improvements to light:

• GitHub Input Validation: As mentioned, strict regex patterns for owner and repo names (^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?$ and ^[a-zA-Z0-9._-]+$, respectively) protect against malformed inputs.
• Metadata Sanitization: We no longer store internal sourcePath in metadata, instead using a generic source: "filesystem", preventing any potential information leakage.
• TypeScript Iteration Fix: A subtle but important fix for zip processing involved using Array.from() for Set iteration, avoiding a TypeScript TS2802 error related to --downlevelIteration (a known project quirk we've documented internally).

Under the Hood: Tech & Dependencies

It's worth noting that these significant features were built without adding any new external dependencies to our package.json. JSZip was already part of our toolkit. For GitHub authentication, we leveraged our existing "Bring Your Own Key" (BYOK) pattern, resolving the GitHub token securely via resolveGitHubToken(tenantId) from our github-connector.ts. This ensures tenant-specific and secure access to GitHub APIs.

Looking Ahead: What's Next for nyxBook

With these features implemented, our immediate next steps are focused on ensuring stability and preparing for further enhancements:

• Rigorous Testing: Comprehensive end-to-end testing for both GitHub import (e.g., selecting a test repo, verifying content import) and zip upload (e.g., creating a test zip, verifying import).
• Atomic Transactions: Considering wrapping our import database writes in prisma.$transaction() to ensure atomicity, meaning either all data is written successfully or none of it is, preventing partial imports.
• UI Refactoring: While manageable at ~530 lines, the import page could benefit from extracting its contents into separate components for improved maintainability and readability.
• Addressing Legacy Risk: The existing server-path import tRPC procedure, which accepts any filesystem path, was flagged as a path traversal risk. We plan to restrict it to a secure upload directory (/tmp/nyxcore-uploads/) or, more likely, remove it entirely now that GitHub and zip uploads cover the primary use cases.

This development session has been incredibly productive, significantly enhancing nyxBook's capabilities and user experience. We're excited about the possibilities these new import methods unlock for our users, making it easier than ever to bring their creative worlds into nyxBook. Stay tuned for more updates!