nyxcore-systems
8 min read

A Night of Triumphs: Fortifying Superadmin Flows, Database Migrations, and PDF Parsing

A deep dive into a recent development sprint, covering the implementation of robust end-to-end superadmin tests, a bulletproof pgvector-safe database migration script, fixing a tricky PDF parsing bug, and refining our team's knowledge base.

e2e testingplaywrightdatabase migrationprismapgvectorpdf parsingnode.jstypescriptsuperadmindevopsdocumentation

It's 12:30 AM UTC, and the commit message is about to hit the main branch. The glow of a successful development session is still fresh, marking the completion of several critical tasks that collectively elevate our platform's stability, security, and developer experience. This wasn't just about shipping features; it was about building a stronger foundation.

Let's unpack the evening's achievements.

Fortifying the Foundation: E2E Superadmin Tests

One of the cornerstones of a robust application is a well-tested administrative interface. Superadmin flows, by their nature, touch sensitive data and critical configurations. Any regression here can have significant consequences. My primary goal for this session was to blanket these flows with comprehensive end-to-end (E2E) tests using Playwright.

We started by extending our tests/e2e/helpers/auth.ts to include dedicated functions for injecting superadmin and tenant-specific session cookies. This refactoring led to a cleaner, more reusable injectSessionCookie() base, making it effortless to simulate various user roles in our tests.

typescript
// Simplified example from tests/e2e/helpers/auth.ts
async function injectSessionCookie(page: Page, sessionToken: string) {
  await page.context().addCookies([
    {
      name: 'next-auth.session-token',
      value: sessionToken,
      domain: 'localhost', // Or your test domain
      path: '/',
      httpOnly: true,
      secure: process.env.NODE_ENV === 'production',
    },
  ]);
  // Optionally, navigate to a page that reloads the session
  await page.goto('/dashboard');
}

export async function injectSuperAdminCookie(page: Page) {
  // Logic to get superadmin session token (e.g., from an API call or fixture)
  const superAdminToken = 'superadmin-jwt-token-here';
  await injectSessionCookie(page, superAdminToken);
}

export async function injectTenantCookie(page: Page, tenantId: string) {
  // Logic to get tenant-specific session token
  const tenantToken = `tenant-${tenantId}-jwt-token-here`;
  await injectSessionCookie(page, tenantToken);
}

With these helpers in place, I crafted tests/e2e/superadmin.spec.ts. This single file now houses 16 distinct tests, each running across both Chromium and mobile viewports, totaling 32 new E2E checks. These tests rigorously validate:

  • Tenant Switcher: Ensuring visibility, seamless switching between tenants, correct checkmark indicators, and the "Other tenants" section behavior.
  • Superadmin Page: Verifying tab access, proper "access denied" scenarios for non-superadmins, full CRUD operations for tenants, user list management, and the invitation dialog flow.
  • Data Isolation: Crucially, we confirmed that switching tenants correctly triggers a data mutation and that non-member tenants correctly display a "join" option, preventing accidental data exposure.

A small but significant improvement for testability was adding data-testid="tenant-switcher" to src/components/layout/tenant-switcher.tsx. This provides a stable, resilient hook for Playwright selectors, decoupling tests from fragile CSS class names.

Database Migrations: Made Safe and Sound

Database migrations are often a source of anxiety, especially in production environments. When dealing with specialized data types like pgvector embeddings, the risk of accidental data loss (e.g., dropping columns) is amplified. To mitigate this, I developed scripts/db-migrate-safe.sh.

This script leverages prisma migrate diff, a powerful tool for generating SQL migration scripts by comparing the current schema to the Prisma schema. The magic happens when it auto-filters any DROP COLUMN operations specifically targeting our sensitive embedding or searchVector columns. This prevents accidental data deletion if a developer inadvertently removes these fields from the Prisma schema.

bash
#!/bin/bash
# scripts/db-migrate-safe.sh

DB_URL=$DATABASE_URL # Ensure this is set in your environment
SAFE_COLUMNS="embedding|searchVector" # Regex for columns to protect

if [ "$1" == "--dry-run" ]; then
  echo "--- DRY RUN: Generated SQL (no changes applied) ---"
  npx prisma migrate diff \
    --from-url "$DB_URL" \
    --to-schema-datamodel prisma/schema.prisma \
    --script \
    | grep -vE "ALTER TABLE .* DROP COLUMN ($SAFE_COLUMNS)"
elif [ "$1" == "--apply" ]; then
  echo "--- APPLYING MIGRATION TO PRODUCTION ---"
  # Generate and filter the migration script
  MIGRATION_SQL=$(npx prisma migrate diff \
    --from-url "$DB_URL" \
    --to-schema-datamodel prisma/schema.prisma \
    --script \
    | grep -vE "ALTER TABLE .* DROP COLUMN ($SAFE_COLUMNS)")

  if [ -z "$MIGRATION_SQL" ]; then
    echo "No safe migrations to apply."
  else
    echo "$MIGRATION_SQL" | psql "$DB_URL"
    echo "Migration applied. Re-running RLS policies..."
    psql "$DB_URL" -f ./prisma/rls.sql
    echo "RLS policies re-applied."
  fi
else
  echo "Usage: $0 [--dry-run | --apply]"
  exit 1
fi

The script supports both --dry-run to preview the filtered SQL and --apply to execute it. A crucial final step is re-running rls.sql after applying changes. This ensures our Row-Level Security policies are always up-to-date and correctly enforced, especially important when schema changes might affect table permissions.

Taming the Wild West of PDF Parsing

Integrating powerful browser-centric libraries into a Node.js server environment can sometimes feel like trying to fit a square peg in a round hole. This session presented one such challenge with PDF parsing.

Our src/server/services/rag/document-processor.ts utilizes pdf-parse@2.4.5, which internally pulls in pdfjs-dist@5.4.296. The issue? pdfjs-dist in this version expects DOMMatrix to be defined in the global scope, a Web API typically found only in browsers. In a Node.js environment, this led to a ReferenceError: DOMMatrix is not defined.

The workaround was to polyfill globalThis.DOMMatrix with a minimal stub that provides an identity matrix before pdfjs-dist is dynamically imported. This allowed the library to initialize without error, and crucially, PDF text extraction now succeeds reliably in our Node.js server.

typescript
// src/server/services/rag/document-processor.ts (simplified)

// Polyfill DOMMatrix for pdfjs-dist in Node.js
if (typeof globalThis.DOMMatrix === 'undefined') {
  class DOMMatrixStub {
    a = 1; b = 0; c = 0; d = 1; e = 0; f = 0; // Identity matrix
    // Add other methods that pdfjs-dist might call, if necessary.
    // For basic text extraction, this minimal stub is often enough.
    translate() { return this; }
    scale() { return this; }
    // ... other stub methods
  }
  globalThis.DOMMatrix = DOMMatrixStub as any;
}

// Dynamic import of pdf-parse to ensure polyfill is in place first
const pdfParse = await import('pdf-parse');

// ... rest of your PDF processing logic

Sharpening Our Tools & Knowledge Base

Beyond direct code changes, a significant part of this session was dedicated to improving our internal documentation and knowledge sharing. All "durable patterns" – recurring solutions, best practices, and critical insights – previously scattered in MEMORY.md were meticulously merged into our central CLAUDE.md. This includes:

  • Best practices for Prisma JSON fields.
  • Persona scope rules and their implementation.
  • Patterns for interacting with the GitHub API.
  • Common test gotchas and how to avoid them.
  • Production deployment checklists.
  • Our standard development workflow.

This consolidation means CLAUDE.md is now a richer, more comprehensive resource, while MEMORY.md has been slimmed down to just open tasks and quick file paths, serving as a dynamic scratchpad rather than a knowledge repository.

Finally, I documented a newly learned skill: ~/.claude/skills/learned/nyxcore-playwright-strict-mode.md. This personal knowledge base entry captures specific Playwright strict mode workarounds, ensuring these hard-won lessons aren't forgotten.

Challenges & Lessons Learned

No development session is without its hurdles. Here's a look at the "pain points" encountered and the valuable lessons they provided:

Playwright's Strict Mode: A Double-Edged Sword

Playwright's strict mode is fantastic for catching ambiguous selectors, but it can be frustrating when multiple elements genuinely share the same text content.

  • Problem: Using getByText('nyxCore'), getByText('Clarait'), or getByText('superadmin') repeatedly led to strict mode violations. The issue was that these text fragments appeared in the sidebar, header, and main content area, making the selector ambiguous.
  • Initial Attempts: Trying to be too generic with text matching.
  • Solution: The key was to be more specific about the context or role of the element:
    • Scope to page.locator("main"): Confining the search to the main content area often resolved ambiguity.
    • Use { exact: true }: For specific, full-text matches.
    • Leverage getByRole("heading", ...): For titles and headings, this is far more robust than getByText.
    • Scope to [role='dialog']: When interacting with modals or pop-ups, explicitly targeting the dialog element ensures you're working within the correct scope.
typescript
// Before (prone to strict mode violation)
await page.getByText('nyxCore').click();

// After (more robust)
await page.locator('main').getByRole('heading', { name: 'nyxCore', exact: true }).click();

// For a button within a dialog
await page.locator('[role="dialog"]').getByRole('button', { name: 'Confirm' }).click();

Targeting Shadcn Components in Playwright

Shadcn UI components are fantastic, but their implementation using Tailwind CSS can sometimes make direct component targeting tricky in tests.

  • Problem: I initially tried locator("[class*='CardContent']") to find specific shadcn Card components. This failed because shadcn renders Tailwind utility classes (e.g., flex, p-4), not component-specific class names like CardContent.
  • Initial Attempts: Relying on implementation details (CSS classes) that are subject to change or non-existent.
  • Solution: Focus on accessibility attributes and roles. Shadcn components are generally well-behaved in terms of semantic HTML.
    • page.locator("main").getByRole("button", ...).first(): Combining a broader scope (main) with an accessibility role (button) and then using .first() (if multiple buttons exist and you need the first relevant one) proved to be a much more reliable strategy. Adding data-testid attributes is also a prime solution here.

Bridging Browser APIs to Node.js

The DOMMatrix issue highlighted a common pitfall when using libraries that have a browser heritage in a Node.js environment.

  • Problem: pdf-parse pulling pdfjs-dist@5.x which expects DOMMatrix to be globally defined, leading to ReferenceError in Node.js.
  • Initial Attempts: Hoping the library would magically adapt or trying to find a Node.js-specific build (which often don't exist for older versions).
  • Solution: A targeted polyfill. By providing a minimal stub for globalThis.DOMMatrix, we satisfied the library's runtime dependency without needing a full browser DOM implementation. This is a pattern to remember for similar situations where a specific browser API is missing in Node.js.

A Productive Sprint and What's Next

This session concluded with 72 out of 72 E2E tests passing, a significant leap from the 40 tests we had before. Typecheck is clean, and all critical development tasks are marked as complete.

The immediate next steps are clear:

  1. Commit and push all changes.
  2. Deploy to production.
  3. Thoroughly test PDF upload functionality in Axiom on the production environment.
  4. Rotate all mini-RAG related secrets (OpenAI, Anthropic, JWT, encryption, DB password) for enhanced security.
  5. Submit nyxcore.cloud for Google Safe Browsing review.

It was a productive night, laying down robust safeguards and streamlining our development process. Onwards to the next challenge!