A Night of Triumphs: Fortifying Superadmin Flows, Database Migrations, and PDF Parsing
A deep dive into a recent development sprint, covering the implementation of robust end-to-end superadmin tests, a bulletproof pgvector-safe database migration script, fixing a tricky PDF parsing bug, and refining our team's knowledge base.
It's 12:30 AM UTC, and the commit message is about to hit the main branch. The glow of a successful development session is still fresh, marking the completion of several critical tasks that collectively elevate our platform's stability, security, and developer experience. This wasn't just about shipping features; it was about building a stronger foundation.
Let's unpack the evening's achievements.
Fortifying the Foundation: E2E Superadmin Tests
One of the cornerstones of a robust application is a well-tested administrative interface. Superadmin flows, by their nature, touch sensitive data and critical configurations. Any regression here can have significant consequences. My primary goal for this session was to blanket these flows with comprehensive end-to-end (E2E) tests using Playwright.
We started by extending our tests/e2e/helpers/auth.ts to include dedicated functions for injecting superadmin and tenant-specific session cookies. This refactoring led to a cleaner, more reusable injectSessionCookie() base, making it effortless to simulate various user roles in our tests.
// Simplified example from tests/e2e/helpers/auth.ts
async function injectSessionCookie(page: Page, sessionToken: string) {
await page.context().addCookies([
{
name: 'next-auth.session-token',
value: sessionToken,
domain: 'localhost', // Or your test domain
path: '/',
httpOnly: true,
secure: process.env.NODE_ENV === 'production',
},
]);
// Optionally, navigate to a page that reloads the session
await page.goto('/dashboard');
}
export async function injectSuperAdminCookie(page: Page) {
// Logic to get superadmin session token (e.g., from an API call or fixture)
const superAdminToken = 'superadmin-jwt-token-here';
await injectSessionCookie(page, superAdminToken);
}
export async function injectTenantCookie(page: Page, tenantId: string) {
// Logic to get tenant-specific session token
const tenantToken = `tenant-${tenantId}-jwt-token-here`;
await injectSessionCookie(page, tenantToken);
}
With these helpers in place, I crafted tests/e2e/superadmin.spec.ts. This single file now houses 16 distinct tests, each running across both Chromium and mobile viewports, totaling 32 new E2E checks. These tests rigorously validate:
- Tenant Switcher: Ensuring visibility, seamless switching between tenants, correct checkmark indicators, and the "Other tenants" section behavior.
- Superadmin Page: Verifying tab access, proper "access denied" scenarios for non-superadmins, full CRUD operations for tenants, user list management, and the invitation dialog flow.
- Data Isolation: Crucially, we confirmed that switching tenants correctly triggers a data mutation and that non-member tenants correctly display a "join" option, preventing accidental data exposure.
A small but significant improvement for testability was adding data-testid="tenant-switcher" to src/components/layout/tenant-switcher.tsx. This provides a stable, resilient hook for Playwright selectors, decoupling tests from fragile CSS class names.
Database Migrations: Made Safe and Sound
Database migrations are often a source of anxiety, especially in production environments. When dealing with specialized data types like pgvector embeddings, the risk of accidental data loss (e.g., dropping columns) is amplified. To mitigate this, I developed scripts/db-migrate-safe.sh.
This script leverages prisma migrate diff, a powerful tool for generating SQL migration scripts by comparing the current schema to the Prisma schema. The magic happens when it auto-filters any DROP COLUMN operations specifically targeting our sensitive embedding or searchVector columns. This prevents accidental data deletion if a developer inadvertently removes these fields from the Prisma schema.
#!/bin/bash
# scripts/db-migrate-safe.sh
DB_URL=$DATABASE_URL # Ensure this is set in your environment
SAFE_COLUMNS="embedding|searchVector" # Regex for columns to protect
if [ "$1" == "--dry-run" ]; then
echo "--- DRY RUN: Generated SQL (no changes applied) ---"
npx prisma migrate diff \
--from-url "$DB_URL" \
--to-schema-datamodel prisma/schema.prisma \
--script \
| grep -vE "ALTER TABLE .* DROP COLUMN ($SAFE_COLUMNS)"
elif [ "$1" == "--apply" ]; then
echo "--- APPLYING MIGRATION TO PRODUCTION ---"
# Generate and filter the migration script
MIGRATION_SQL=$(npx prisma migrate diff \
--from-url "$DB_URL" \
--to-schema-datamodel prisma/schema.prisma \
--script \
| grep -vE "ALTER TABLE .* DROP COLUMN ($SAFE_COLUMNS)")
if [ -z "$MIGRATION_SQL" ]; then
echo "No safe migrations to apply."
else
echo "$MIGRATION_SQL" | psql "$DB_URL"
echo "Migration applied. Re-running RLS policies..."
psql "$DB_URL" -f ./prisma/rls.sql
echo "RLS policies re-applied."
fi
else
echo "Usage: $0 [--dry-run | --apply]"
exit 1
fi
The script supports both --dry-run to preview the filtered SQL and --apply to execute it. A crucial final step is re-running rls.sql after applying changes. This ensures our Row-Level Security policies are always up-to-date and correctly enforced, especially important when schema changes might affect table permissions.
Taming the Wild West of PDF Parsing
Integrating powerful browser-centric libraries into a Node.js server environment can sometimes feel like trying to fit a square peg in a round hole. This session presented one such challenge with PDF parsing.
Our src/server/services/rag/document-processor.ts utilizes pdf-parse@2.4.5, which internally pulls in pdfjs-dist@5.4.296. The issue? pdfjs-dist in this version expects DOMMatrix to be defined in the global scope, a Web API typically found only in browsers. In a Node.js environment, this led to a ReferenceError: DOMMatrix is not defined.
The workaround was to polyfill globalThis.DOMMatrix with a minimal stub that provides an identity matrix before pdfjs-dist is dynamically imported. This allowed the library to initialize without error, and crucially, PDF text extraction now succeeds reliably in our Node.js server.
// src/server/services/rag/document-processor.ts (simplified)
// Polyfill DOMMatrix for pdfjs-dist in Node.js
if (typeof globalThis.DOMMatrix === 'undefined') {
class DOMMatrixStub {
a = 1; b = 0; c = 0; d = 1; e = 0; f = 0; // Identity matrix
// Add other methods that pdfjs-dist might call, if necessary.
// For basic text extraction, this minimal stub is often enough.
translate() { return this; }
scale() { return this; }
// ... other stub methods
}
globalThis.DOMMatrix = DOMMatrixStub as any;
}
// Dynamic import of pdf-parse to ensure polyfill is in place first
const pdfParse = await import('pdf-parse');
// ... rest of your PDF processing logic
Sharpening Our Tools & Knowledge Base
Beyond direct code changes, a significant part of this session was dedicated to improving our internal documentation and knowledge sharing. All "durable patterns" – recurring solutions, best practices, and critical insights – previously scattered in MEMORY.md were meticulously merged into our central CLAUDE.md. This includes:
- Best practices for Prisma JSON fields.
- Persona scope rules and their implementation.
- Patterns for interacting with the GitHub API.
- Common test gotchas and how to avoid them.
- Production deployment checklists.
- Our standard development workflow.
This consolidation means CLAUDE.md is now a richer, more comprehensive resource, while MEMORY.md has been slimmed down to just open tasks and quick file paths, serving as a dynamic scratchpad rather than a knowledge repository.
Finally, I documented a newly learned skill: ~/.claude/skills/learned/nyxcore-playwright-strict-mode.md. This personal knowledge base entry captures specific Playwright strict mode workarounds, ensuring these hard-won lessons aren't forgotten.
Challenges & Lessons Learned
No development session is without its hurdles. Here's a look at the "pain points" encountered and the valuable lessons they provided:
Playwright's Strict Mode: A Double-Edged Sword
Playwright's strict mode is fantastic for catching ambiguous selectors, but it can be frustrating when multiple elements genuinely share the same text content.
- Problem: Using
getByText('nyxCore'),getByText('Clarait'), orgetByText('superadmin')repeatedly led to strict mode violations. The issue was that these text fragments appeared in the sidebar, header, and main content area, making the selector ambiguous. - Initial Attempts: Trying to be too generic with text matching.
- Solution: The key was to be more specific about the context or role of the element:
- Scope to
page.locator("main"): Confining the search to the main content area often resolved ambiguity. - Use
{ exact: true }: For specific, full-text matches. - Leverage
getByRole("heading", ...): For titles and headings, this is far more robust thangetByText. - Scope to
[role='dialog']: When interacting with modals or pop-ups, explicitly targeting the dialog element ensures you're working within the correct scope.
- Scope to
// Before (prone to strict mode violation)
await page.getByText('nyxCore').click();
// After (more robust)
await page.locator('main').getByRole('heading', { name: 'nyxCore', exact: true }).click();
// For a button within a dialog
await page.locator('[role="dialog"]').getByRole('button', { name: 'Confirm' }).click();
Targeting Shadcn Components in Playwright
Shadcn UI components are fantastic, but their implementation using Tailwind CSS can sometimes make direct component targeting tricky in tests.
- Problem: I initially tried
locator("[class*='CardContent']")to find specific shadcn Card components. This failed because shadcn renders Tailwind utility classes (e.g.,flex,p-4), not component-specific class names likeCardContent. - Initial Attempts: Relying on implementation details (CSS classes) that are subject to change or non-existent.
- Solution: Focus on accessibility attributes and roles. Shadcn components are generally well-behaved in terms of semantic HTML.
page.locator("main").getByRole("button", ...).first(): Combining a broader scope (main) with an accessibility role (button) and then using.first()(if multiple buttons exist and you need the first relevant one) proved to be a much more reliable strategy. Addingdata-testidattributes is also a prime solution here.
Bridging Browser APIs to Node.js
The DOMMatrix issue highlighted a common pitfall when using libraries that have a browser heritage in a Node.js environment.
- Problem:
pdf-parsepullingpdfjs-dist@5.xwhich expectsDOMMatrixto be globally defined, leading toReferenceErrorin Node.js. - Initial Attempts: Hoping the library would magically adapt or trying to find a Node.js-specific build (which often don't exist for older versions).
- Solution: A targeted polyfill. By providing a minimal stub for
globalThis.DOMMatrix, we satisfied the library's runtime dependency without needing a full browser DOM implementation. This is a pattern to remember for similar situations where a specific browser API is missing in Node.js.
A Productive Sprint and What's Next
This session concluded with 72 out of 72 E2E tests passing, a significant leap from the 40 tests we had before. Typecheck is clean, and all critical development tasks are marked as complete.
The immediate next steps are clear:
- Commit and push all changes.
- Deploy to production.
- Thoroughly test PDF upload functionality in Axiom on the production environment.
- Rotate all mini-RAG related secrets (OpenAI, Anthropic, JWT, encryption, DB password) for enhanced security.
- Submit
nyxcore.cloudfor Google Safe Browsing review.
It was a productive night, laying down robust safeguards and streamlining our development process. Onwards to the next challenge!