Developer's Log: Taming Superadmin Flows, pgvector Migrations, and Edge Runtime Quirks

A Note to Future Me (and Fellow Devs)

Sometimes, the best way to hand off a complex session – even to your future self – is to document the journey, the triumphs, and especially, the landmines. This log isn't just a status update; it's a narrative of exploration, problem-solving, and a critical design phase for some core features in our system.

Today’s mission was clear: lay the groundwork for end-to-end (E2E) testing of our superadmin flows, devise a safe production migration strategy for our pgvector embeddings, and ensure our tenant switcher UX is rock solid. The design phase is now complete, and with user approval pending, we're on the cusp of implementation.

The Mission Brief: What We Set Out To Do

Our system relies heavily on a robust superadmin interface for managing tenants and critical configurations. Ensuring these flows are bulletproof through E2E tests is paramount. Simultaneously, our use of pgvector for AI-powered insights introduces a unique challenge: how do we safely evolve our database schema when Prisma, by default, doesn't natively understand vector types? And finally, a smooth tenant switching experience is key to the superadmin's day-to-day.

Diving In: Explorations and Discoveries

Before writing a single line of new feature code, a thorough exploration of the existing codebase was essential.

1. E2E Test Setup with Playwright: We dove deep into our existing Playwright configuration, understanding how our tests/e2e/helpers/auth.ts helper injects JWT cookies. The plan is to extend this with an injectSuperAdminCookie() helper, allowing our E2E tests to impersonate a superadmin user. Crucially, we've established a pattern for mocking tRPC calls using Playwright's network interception, as seen in tests/e2e/compliance-export.spec.ts. This allows us to test the UI and client-side logic in isolation, even for complex superjson-wrapped tRPC responses.

   typescript

   // Conceptual snippet for auth.ts helper
   export async function injectSuperAdminCookie(page: Page) {
     const jwtPayload = {
       sub: "e2e-superadmin-id",
       tenantId: "e2e-global-tenant-id", // Or null/special ID for global superadmin
       role: "superadmin",
     };
     // Logic to sign and set the JWT as a cookie
     await page.context().addCookies([
       {
         name: 'next-auth.session-token',
         value: await signJwt(jwtPayload), // Assuming a signJwt helper exists
         domain: 'localhost', // Or your app's domain
         path: '/',
       },
     ]);
   }
   
   // Conceptual snippet for tRPC mocking in a spec file
   await page.route('/api/trpc/**', async (route) => {
     if (route.request().postDataJSON()?.['0'].path === 'superadmin.getDashboardData') {
       await route.fulfill({
         status: 200,
         contentType: 'application/json',
         body: JSON.stringify({
           // Superjson wrapped response for mocked data
           json: { data: { /* mocked superadmin data */ } },
           meta: { /* superjson meta */ }
         }),
       });
     } else {
       await route.continue();
     }
   });
   

2. Superadmin & Tenant Switching Mechanics: We mapped out the core components: src/components/layout/tenant-switcher.tsx, the src/server/trpc/routers/superadmin.ts and src/server/trpc/routers/teams.ts tRPC routers, and how our auth.ts JWT callback handles roles and tenant context. The goal is to ensure that when a superadmin switches tenants, the UI correctly reflects the new context, and more importantly, data isolation is maintained.

3. Prisma & pgvector - A Delicate Dance: Our setup uses pgvector for embedding storage, with vector columns defined directly in rls.sql rather than in the Prisma schema, and HNSW indexes managed separately. This is because Prisma's current version (@prisma/client@5.22.0) doesn't natively support vector types, marking them as Unsupported. Our database schema is currently managed via prisma db push in development, but this approach has a critical flaw for production.

The Gauntlet: Lessons Learned and Pain Points

This session wasn't just about planning; it was about uncovering crucial issues before they become production nightmares.

Lesson 1: The pgvector Migration Minefield

This is perhaps the most critical takeaway. Our workflow_insights and document_chunks tables contain embedding vector(1536) columns. When running prisma db push against a production database that already has these columns, Prisma's introspection sees them as Unsupported and, in its infinite wisdom, proposes to DROP these columns.

• The Danger: Running prisma db push --accept-data-loss on production would unequivocally destroy all our vector embedding data, which is foundational to our AI features. This is an absolute NEVER DO.

• The Workaround (Planned): We need a safe, controlled migration strategy. The plan is to create a scripts/db-migrate-safe.sh script that will:

  1. Use prisma migrate diff --from-schema-datasource --to-schema-datasource --script to generate the raw SQL for schema changes.
  2. Crucially, filter out any SQL statements that attempt to DROP embedding vector(1536) columns. This is the core safety mechanism.
  3. Apply the filtered SQL changes using psql.
  4. Re-run our rls.sql script to ensure all Row-Level Security policies and custom pgvector indexes (like HNSW) are correctly applied or recreated.

  This ensures that while Prisma manages the parts of the schema it understands, we manually safeguard our pgvector columns. We're using pgvector/pgvector:pg16 Docker image consistently across dev and prod, which is good for consistency.

Lesson 2: Taming the Edge Runtime

A prior session (commit 54933e0) taught us a valuable lesson: src/server/auth.ts runs in the Edge Runtime. This means NEVER import Node-only modules (like crypto, fs, path, or even some third-party libraries that rely on them) into this file. Doing so will lead to frustrating runtime errors that are hard to debug in a serverless environment. This is a constant reminder for any file intended for Edge deployment.

Current State & The Path Forward

Our production server, root@46.225.232.35, is currently at commit 54933e0 – the very one that solidified our understanding of Edge Runtime constraints.

The next steps are clear and focused:

1. [ ] Get user approval on design proposal (3 tasks): This is the gate before implementation.
2. [ ] Create tests/e2e/superadmin.spec.ts: Start building the actual E2E tests for superadmin flows, leveraging tRPC mocking.
3. [ ] Extend tests/e2e/helpers/auth.ts with injectSuperAdminCookie(): Implement the helper to enable superadmin context in tests.
4. [ ] Create scripts/db-migrate-safe.sh: Develop and test the pgvector-safe migration script. This is a critical piece of infrastructure.
5. [ ] Run E2E tests to verify (npm run test:e2e): Ensure everything works as expected.
6. [ ] Tenant switcher UX covered by superadmin E2E tests: Confirm that tenant switching and data isolation are robustly tested.

This session has been invaluable. It's not just about planning features, but about proactively identifying and mitigating significant technical debt and potential production issues. The path ahead is challenging, but with a clear plan and lessons learned, we're ready.

{
  "thingsDone": [
    "Explored full E2E test setup (Playwright config, auth helper, existing spec files)",
    "Explored tenant switcher/superadmin components (UI, tRPC routers, auth JWT callback)",
    "Explored Prisma/pgvector setup (db push, rls.sql for vector columns, HNSW indexes)",
    "Designed a 3-task implementation plan for E2E, migrations, and UX",
    "Presented design proposal to user for approval"
  ],
  "pains": [
    "Prisma's 'Unsupported' type for pgvector columns leading to DROP proposals in `db push`",
    "Danger of `prisma db push --accept-data-loss` on production with pgvector data",
    "Edge Runtime limitations requiring vigilance on Node-only module imports in auth.ts"
  ],
  "successes": [
    "Identified critical pgvector migration issue *before* production deployment",
    "Designed a safe, manual migration strategy using `prisma migrate diff` and SQL filtering",
    "Established a clear E2E testing strategy including tRPC mocking and auth helpers",
    "Reinforced understanding of Edge Runtime constraints for critical serverless functions"
  ],
  "techStack": [
    "Playwright",
    "Prisma",
    "pgvector",
    "tRPC",
    "Next.js",
    "TypeScript",
    "PostgreSQL",
    "Docker",
    "Edge Runtime",
    "JWT"
  ]
}