← nyxcore-systems
8 min read

From Session Notes to Production: E2E, PGVector, and the DOMMatrix Odyssey

Join me on a recent development journey covering critical E2E tests, a tricky pgvector migration, a peculiar PDF parsing fix, and essential knowledge management. A look behind the curtain of a real-world dev session.

PlaywrightE2E TestingPostgreSQLpgvectorPrismaPDF.jsNode.jsTypeScriptDevOpsMigrationTroubleshooting

Every now and then, I find myself in a development session that feels like a mini-sprint in itself. It’s a focused block of time where multiple threads converge, problems are debugged, features are shipped, and the mental model of the system solidifies a little more. I often capture these sessions as "Letters to Myself" – a raw dump of goals, successes, pains, and immediate next steps.

This post is a public-ready version of one such recent session (dated 2026-03-04), where I tackled everything from beefing up end-to-end tests to wrestling with database migrations and a peculiar browser API in Node.js. It’s a glimpse into the nitty-gritty of getting things done and the lessons learned along the way.

The Mission Brief: What Was On My Plate

The primary goal for this session was to push a set of critical updates and fixes to production. This included:

  • Robust E2E Tests: Specifically for our superadmin user flow, ensuring core functionalities are locked down.
  • pgvector-Safe Database Migration: A tricky one, ensuring Prisma migrations don't inadvertently drop our precious vector embeddings.
  • PDF Processing Fix: Addressing a DOMMatrix error when parsing PDFs in Node.js.
  • Knowledge Consolidation: Merging durable development patterns into our central knowledge base.

By the end of the session, all these items were not just done, but committed (9d0eb60), deployed to production, and confirmed healthy. Phew!

The "Done" List: A Deep Dive into the Accomplishments

Let's break down what got shipped and why it mattered.

1. Superadmin E2E Tests with Playwright

Building robust systems means ensuring critical paths are always working. For a superadmin user, this is paramount.

  • tests/e2e/superadmin.spec.ts: I added 16 new tests, running across both Chromium and mobile viewports, totaling 32 new E2E checks. This brought our total E2E test count from 40 to a solid 72, all passing.
  • tests/e2e/helpers/auth.ts: To facilitate testing different user roles, I refactored our authentication helpers. Now we have dedicated injectSuperAdminCookie() and injectTenantCookie() functions, both building on a shared injectSessionCookie() base. This makes our tests cleaner and more maintainable.
  • src/components/layout/tenant-switcher.tsx: A small but crucial change: adding data-testid="tenant-switcher" to a key UI element. This makes it far easier for Playwright to reliably target and interact with the tenant switcher, regardless of future styling changes.

2. The pgvector-Safe Migration Script

This was one of the more nail-biting tasks. When working with pgvector in PostgreSQL, we store vector embeddings in a dedicated column type. Prisma, while powerful, has a known behavior where it might try to drop columns it doesn't explicitly manage during a migration, especially if they're of a type it doesn't natively understand (like vector).

To circumvent this, I developed a custom, pgvector-safe migration script: scripts/db-migrate-safe.sh.

Here's the high-level workflow:

  1. Generate Diff: prisma migrate diff --from-url $DATABASE_URL --to-schema-datamodel $SCHEMA_PATH --script > migration.sql
  2. Filter Vector DROPs: Crucially, I pipe this migration.sql through a filter (e.g., grep -v 'DROP TABLE.*vector_table') to remove any commands that would destroy our vector data.
  3. Execute Safely: The filtered SQL is then applied directly via psql.
  4. Re-run RLS: After the migration, I re-run our Row-Level Security (RLS) policies to ensure they are correctly applied to any new or altered tables.

This ensures our vector embeddings remain untouched while still allowing Prisma to manage other schema changes. A critical piece of infrastructure when dealing with hybrid database schema management.

3. DOMMatrix Polyfill for pdfjs-dist in Node.js

We use pdf-parse which, under the hood, leverages pdfjs-dist for PDF processing. After an update, I started seeing DOMMatrix is not defined errors in our Node.js environment.

The culprit? pdfjs-dist@5.x dropped its legacy build, meaning it now assumes a browser-like environment where DOMMatrix is globally available. Node.js, naturally, doesn't have it.

My fix involved a minimal polyfill:

typescript
// src/server/services/rag/document-processor.ts
// ... other imports
if (typeof globalThis.DOMMatrix === 'undefined') {
  globalThis.DOMMatrix = class DOMMatrix {
    a = 1; b = 0; c = 0; d = 1; e = 0; f = 0;
    // Basic stub for methods pdfjs-dist might call
    scale(scaleX: number, scaleY: number) {
      this.a *= scaleX; this.d *= scaleY; return this;
    }
    translate(tx: number, ty: number) {
      this.e += tx; this.f += ty; return this;
    }
    // Add other methods as needed based on specific pdfjs-dist usage
  } as any; // Use 'any' to satisfy type checker for basic stub
}

// Dynamically import pdf-parse AFTER the polyfill
const pdfParse = (await import('pdf-parse')).default;
// ... rest of your PDF processing logic

This simple stub provides just enough of a DOMMatrix implementation to satisfy pdfjs-dist without needing a heavy browser emulation library. Verified working on local and now in production!

4. Knowledge Management: MEMORY.md to CLAUDE.md

As a system evolves, so does the institutional knowledge around it. I merged all durable patterns, gotchas, and best practices from MEMORY.md (which was becoming a bit of a chaotic brain dump) into CLAUDE.md.

CLAUDE.md now serves as our canonical knowledge base, containing:

  • Persona rules
  • Deployment strategies
  • Prisma-specific gotchas
  • E2E test patterns
  • GitHub API integration notes

MEMORY.md has been slimmed down to just open tasks and quick paths, making it more of an ephemeral scratchpad. This consolidation improves discoverability and ensures critical knowledge isn't lost. I also captured a new "learned skill" for nyxcore-playwright-strict-mode.md, documenting the best practices for Playwright locators (more on this in the "Pain Log").

The "Pain Log": Lessons Learned and War Stories

Not everything goes smoothly. These are the moments that truly teach you something.

1. Playwright Strict Mode Violations

  • Tried: Using page.getByText('Some Text') for elements that might appear in multiple places (e.g., a common phrase in a sidebar, header, and main content area).
  • Failed: Playwright's strict mode kicked in, complaining about multiple matches. This is a good thing – it forces you to write more precise selectors.
  • Workaround/Lesson: Scope, scope, scope!
    • Specificity: If the text is unique within a section, scope it: page.locator("main").getByText('Some Text').
    • Exactness: Use { exact: true } when matching text to avoid partial matches.
    • Role-based: Prefer getByRole("heading", { name: "Dashboard" }) for headings or other semantic elements.
    • data-testid: As mentioned, adding data-testid attributes is often the most robust solution for interactive elements.

Takeaway: Embrace Playwright's strict mode. It's a guardian against flaky tests. When it complains, it's telling you your selector isn't precise enough for the element you intend to interact with.

2. The DOMMatrix Saga (Already Covered)

This was a classic "library updated, broke in Node.js" scenario. The lesson here is to always be wary of major version bumps, especially for libraries that have a browser-centric origin. Checking changelogs for dropped legacy support or new browser API dependencies is crucial.

3. Shadcn/Tailwind Class Name Gotchas

  • Tried: Using locator("[class*='CardContent']") to target a Shadcn CardContent component.
  • Failed: This selector did NOT work.
  • Lesson: Shadcn UI components are built with Tailwind CSS. Tailwind generates highly specific, often hashed, utility classes (e.g., flex items-center justify-between space-x-2 rounded-md border p-4 shadow-sm). You cannot reliably target components by trying to find substrings of their component name within their generated class attribute. The class attribute will contain Tailwind utility classes, not CardContent.

Takeaway: For E2E tests, never rely on generated CSS classes (especially from utility-first frameworks like Tailwind) for targeting components. Always use stable attributes like data-testid or semantic roles (getByRole).

What's Next: The Road Ahead

Even after a productive session, the backlog never truly empties. Here are the immediate next steps:

  1. Verify PDF Upload in Production: Confirm the DOMMatrix fix works flawlessly within our containerized production environment by testing actual PDF uploads.
  2. Rotate Secrets: A critical security practice – rotate all sensitive secrets (OpenAI, Anthropic, JWT, encryption keys, DB password).
  3. Google Safe Browsing Review: Manually submit nyxcore.cloud for review to ensure it's not flagged.
  4. E2E Test: Compliance Export: Add an E2E test to verify the compliance report export button correctly triggers a .md download.
  5. E2E Test: PR Checkbox & Linked Repo: Extend E2E coverage to verify GitHub integration for PR checkboxes and linked repositories.
  6. Update Documentation: Reflect the new compliance report export functionality in docs/06-workflow-intelligence.md.

Conclusion

This session was a microcosm of full-stack development: testing, database ops, tricky environment-specific polyfills, and continuous knowledge refinement. It's a reminder that shipping features isn't just about writing new code, but about ensuring stability, addressing technical debt, and learning from every hiccup along the way.

What are your recent "Letter to Myself" development sessions like? Share your war stories and lessons learned in the comments!

json
{
  "thingsDone": [
    "Implemented 32 new E2E tests for superadmin flows using Playwright",
    "Refactored E2E authentication helpers for superadmin and tenant roles",
    "Developed a custom pgvector-safe database migration script",
    "Implemented a minimal DOMMatrix polyfill for pdfjs-dist@5.x in Node.js",
    "Consolidated durable development patterns from MEMORY.md into CLAUDE.md",
    "Added data-testid to tenant-switcher for robust E2E targeting"
  ],
  "pains": [
    "Playwright strict mode violations due to ambiguous getByText() locators",
    "DOMMatrix is not defined error in Node.js with pdfjs-dist@5.x",
    "Inability to reliably target Shadcn components using class*='ComponentName' selectors"
  ],
  "successes": [
    "Achieved 72/72 passing E2E tests after session",
    "Successfully deployed pgvector-safe migration to production without data loss",
    "Resolved PDF parsing issue with a minimal polyfill",
    "Improved E2E test reliability and maintainability",
    "Centralized and organized development knowledge base"
  ],
  "techStack": [
    "Playwright",
    "TypeScript",
    "Node.js",
    "PostgreSQL",
    "pgvector",
    "Prisma",
    "pdfjs-dist",
    "pdf-parse",
    "Shadcn UI",
    "Tailwind CSS",
    "GitHub Actions (implied by E2E/CI)"
  ]
}