← nyxcore-systems
8 min read

Late Night Dev: Conquering Superadmin E2E, pgvector Migrations, and the Elusive DOMMatrix

A deep dive into a late-night dev session tackling critical E2E tests, crafting pgvector-safe database migrations, and wrestling with browser APIs in Node.js.

e2e-testingplaywrightprismapgvectorpdf-parsingnode.jstypescriptdevopsfullstacklessons-learned

It's 00:30 UTC, the world is quiet, and the only sound is the hum of my machine. This is often when some of the most focused and impactful development happens. Tonight was one of those sessions – a deep dive into critical system areas, from end-to-end testing superadmin flows to bulletproofing database migrations and even taming browser APIs in a Node.js environment.

The goal was ambitious: get full E2E coverage for our superadmin features, build a robust migration script for our pgvector-enabled database, fix a pesky PDF parsing bug, and consolidate some crucial institutional knowledge. By the end, all greens, all deployed. Let's break down the journey and the lessons learned.

Mission Critical: Superadmin E2E Tests

Our application, nyxCore, has a powerful superadmin interface. Ensuring its stability and, crucially, its data isolation properties, is paramount. This meant building out a comprehensive suite of E2E tests.

I started by extending our Playwright helpers in tests/e2e/helpers/auth.ts. We now have dedicated functions like injectSuperAdminCookie() and injectTenantCookie(), all refactored to use a shared injectSessionCookie() base. This makes setting up authenticated test contexts a breeze.

Then came the beast: tests/e2e/superadmin.spec.ts. This single file now houses 16 distinct tests, each running on both Chromium and mobile viewports, totaling 32 new passing tests. These cover:

  • Tenant Switcher: Ensuring visibility, correct switching logic, check marks, and proper display of "Other tenants."
  • Superadmin Page: Validating tab navigation, access denial for non-superadmins, full CRUD operations for tenants, user list management, and the invitation flow.
  • Data Isolation: Crucially, verifying that switching tenants correctly triggers data mutations and that non-member tenants correctly show "join" options, preventing accidental data leakage.

Playwright Pains & Hard-Won Lessons

Building these tests wasn't without its challenges, particularly when dealing with Playwright's strict mode and the nuances of component libraries like shadcn/ui.

Lesson 1: Embrace Specificity with Playwright Strict Mode

Initially, I found myself battling Playwright's strict mode errors. Using broad selectors like getByText('nyxCore'), getByText('Clarait'), or getByText('superadmin') often led to violations because these text snippets appeared in multiple places (sidebar, header, main content).

  • The Problem: Playwright, by default, expects selectors to uniquely identify an element. If multiple elements contain the text, it throws a strict mode error.
  • The Fix: Scope your selectors aggressively.
    • Scope to the main content area: page.locator("main").getByText("...")
    • Use exact: true for precise matches: getByText("Superadmin", { exact: true })
    • Leverage semantic roles: getByRole("heading", { name: "Tenant Management" }) or page.locator("[role='dialog']").getByRole("button", { name: "Invite" }) are much more robust.

Lesson 2: Shadcn/ui Components Don't Play Nice with Class Selectors

I tried to target shadcn components using class-based selectors, like locator("[class*='CardContent']").

  • The Problem: Shadcn components primarily render Tailwind CSS utility classes, not component-specific class names. [class*='CardContent'] wouldn't reliably find the intended element because the actual class names are things like p-6 flex flex-col space-y-1.5.
  • The Fix: Rely on semantic HTML and roles provided by shadcn. If a shadcn Card contains a button, target it via its role within a scoped area: page.locator("main").getByRole("button", { name: "Add Tenant" }) is far more reliable. Add data-testid attributes to your components for explicit targeting if semantic roles aren't sufficient. I added data-testid="tenant-switcher" to our tenant-switcher.tsx for this exact reason.

Bulletproofing Production Migrations: The pgvector Conundrum

Database migrations are always a tense moment, especially in production. We use pgvector for embedding and search vector columns, and these have a critical caveat: you cannot DROP COLUMN on them if they are part of an index or have associated data. This makes standard prisma migrate deploy potentially dangerous if a schema change tries to drop such a column.

My solution was scripts/db-migrate-safe.sh. This script leverages prisma migrate diff to generate the SQL, but then intelligently filters out problematic DROP COLUMN statements related to embedding or searchVector columns.

bash
#!/bin/bash
# scripts/db-migrate-safe.sh

# ... (setup for PRISMA_SCHEMA_PATH, DATABASE_URL, etc.)

# Generate the migration SQL
SQL_DIFF=$(npx prisma migrate diff \
  --from-url "$DATABASE_URL" \
  --to-schema-datamodel "$PRISMA_SCHEMA_PATH" \
  --script \
  --preview-data-source)

# Filter out dangerous DROP COLUMN statements for pgvector columns
FILTERED_SQL=$(echo "$SQL_DIFF" | grep -vE 'ALTER TABLE "([a-zA-Z0-9_]+)" DROP COLUMN ("embedding"|"searchVector")')

if [[ "$1" == "--dry-run" ]]; then
  echo "--- Dry Run Migration SQL (filtered for pgvector safety) ---"
  echo "$FILTERED_SQL"
  exit 0
fi

if [[ "$1" == "--apply" ]]; then
  if [[ -z "$FILTERED_SQL" ]]; then
    echo "No pending migrations or all dangerous DROP COLUMNs were filtered. Database is up to date."
  else
    echo "Applying filtered migration..."
    echo "$FILTERED_SQL" | psql "$DATABASE_URL" -v ON_ERROR_STOP=1
    echo "Migration applied. Re-running RLS script..."
    psql "$DATABASE_URL" -f "./prisma/rls.sql" -v ON_ERROR_STOP=1
    echo "RLS script re-applied successfully."
  fi
  exit 0
fi

echo "Usage: $0 [--dry-run | --apply]"
exit 1

This script offers --dry-run to inspect the changes and --apply to execute them safely. A crucial detail: it re-runs our rls.sql script after applying changes to ensure Row Level Security policies are always up-to-date with any new tables or column changes. This is a robust pattern for production deployments when dealing with specialized database extensions.

Taming PDFs in Node.js: The DOMMatrix Saga

Our RAG (Retrieval Augmented Generation) pipeline relies on extracting text from various document types, including PDFs. Suddenly, our PDF parsing started failing in Node.js with a ReferenceError: DOMMatrix is not defined.

  • The Problem: We're using pdf-parse@2.4.5, which pulls in pdfjs-dist@5.4.296. This version of pdfjs-dist has a dependency on DOMMatrix, a browser-specific API, even when running in a Node.js environment.
  • The Fix: Polyfill globalThis.DOMMatrix with a minimal stub that provides an identity matrix. This satisfies pdfjs-dist's runtime check without needing a full browser environment.
typescript
// src/server/services/rag/document-processor.ts
import { dynamicImport } from '@/lib/utils/dynamic-import';

// Polyfill DOMMatrix for pdfjs-dist in Node.js
// pdfjs-dist@5.x requires DOMMatrix, which is a browser API.
// Provide a minimal stub to prevent ReferenceError.
if (typeof globalThis.DOMMatrix === 'undefined') {
  globalThis.DOMMatrix = class DOMMatrix {
    a = 1; b = 0; c = 0; d = 1; e = 0; f = 0; // Identity matrix
    constructor(init?: string | number[]) {
      // Basic constructor for compatibility, not full implementation
      if (typeof init === 'string') {
        // Parse CSS transform string if needed, or just ignore for stub
      } else if (Array.isArray(init)) {
        [this.a, this.b, this.c, this.d, this.e, this.f] = init;
      }
    }
    // Add other methods that pdfjs-dist might call if necessary, e.g.,
    // invertSelf() { return this; }
    // multiply(other: DOMMatrix) { return this; }
  } as any; // Cast to any to satisfy type checker for partial implementation
}

// Dynamically import pdf-parse to ensure polyfill is in place first
const pdfParse = dynamicImport('pdf-parse');

// ... rest of PDF processing logic

After implementing this polyfill, PDF text extraction immediately sprang back to life. A small but critical fix for our RAG pipeline.

Housekeeping & Knowledge Consolidation

Beyond the core features and fixes, this session also focused on knowledge management. I merged all our durable patterns and architectural decisions from MEMORY.md into CLAUDE.md. CLAUDE.md is now our definitive, living document for:

  • Best practices for Prisma JSON fields
  • Persona scope rules and their implementation
  • Patterns for GitHub API integration
  • Common Playwright test gotchas (now with more lessons!)
  • Production deployment strategies
  • Our standard dev workflow

MEMORY.md has been slimmed down to just open tasks and quick file paths – a scratchpad rather than a knowledge base. I also captured the Playwright strict mode learnings into a new skill file: ~/.claude/skills/learned/nyxcore-playwright-strict-mode.md, ensuring this hard-won knowledge is easily retrievable for future sessions.

The Triumphant Finish

As the clock ticked past 2 AM, the dashboard showed 72/72 E2E tests passing (up from 40 before the session), typecheck clean, and all changes committed. The feeling of pushing these critical updates to production, knowing the system is more robust and secure, is incredibly rewarding.

Immediate next steps:

  1. Deploy to production (done!)
  2. Test PDF upload in Axiom on production (verify the fix end-to-end)
  3. Rotate mini-rag secrets (a routine security measure after significant deployments)
  4. Submit nyxcore.cloud for Google Safe Browsing review (another crucial security step)

This session highlights the multi-faceted nature of full-stack development. It's not just about writing new features, but about ensuring quality through robust testing, safeguarding deployments with careful scripting, and maintaining a healthy, documented codebase. Each challenge, from Playwright selectors to browser APIs in Node.js, offers a valuable lesson that ultimately strengthens the entire system.

json
{
  "thingsDone": [
    "Extended E2E test helpers for superadmin/tenant authentication",
    "Created 32 new E2E tests for superadmin flows (tenant switcher, CRUD, data isolation)",
    "Developed pgvector-safe database migration script (db-migrate-safe.sh)",
    "Fixed PDF parsing 'DOMMatrix is not defined' error with a polyfill",
    "Added data-testid to tenant switcher component",
    "Merged durable patterns from MEMORY.md into CLAUDE.md",
    "Slimmed MEMORY.md to open tasks",
    "Created new Playwright strict mode learned skill markdown"
  ],
  "pains": [
    "Playwright strict mode violations due to overlapping text selectors",
    "Failed shadcn component selection using class-based selectors",
    "ReferenceError: DOMMatrix is not defined when parsing PDFs in Node.js"
  ],
  "successes": [
    "Robust Playwright selectors using scoping, exact matches, and roles",
    "Semantic element selection for shadcn components instead of class selectors",
    "Polyfilling DOMMatrix in Node.js for pdfjs-dist compatibility",
    "Automated safe pgvector migrations with dry-run and apply options"
  ],
  "techStack": [
    "Playwright",
    "Prisma",
    "pgvector",
    "Node.js",
    "pdfjs-dist",
    "pdf-parse",
    "shadcn/ui",
    "Tailwind CSS",
    "TypeScript",
    "Bash",
    "PostgreSQL",
    "Axiom"
  ]
}