Building a Dynamic Workflow Engine: When Prisma Types Fight Back

Building complex systems often means wrestling with the tools we rely on. Today, I want to share the journey of creating a dynamic workflow builder and execution engine, complete with AI integration and project consolidation features. Along the way, we'll explore some fascinating Prisma ORM quirks that taught me valuable lessons about type systems and database relationships.

The Vision: Workflows That Think

The goal was ambitious: create a system where users can build custom workflows with drag-and-drop simplicity, then execute them with real-time streaming updates. Think GitHub Actions meets Zapier, but with AI-powered steps that can reference project knowledge.

Key features included:

• Dynamic workflow builder with visual step composition
• Real-time execution engine with SSE streaming
• AI integration with configurable LLM providers
• Project wisdom injection via consolidation data
• Resume/retry capabilities for failed workflows

The Architecture Journey

Phase 1-3: Schema Evolution

The foundation started with extending our Prisma schema. We added 13 new fields to WorkflowStep, introduced a WorkflowTemplate model, and connected workflows to users and consolidations:

model Workflow {
  id              String   @id @default(cuid())
  name            String
  description     String?
  userId          String
  tenantId        String
  consolidationIds String[] // JSON array of linked project data
  steps           WorkflowStep[]
  // ... other fields
}

Phase 2-5: The Engine Rewrite

The execution engine became the heart of the system. We implemented:

class ChainContext {
  // Template variable resolution: {{input}}, {{steps.Label.content}}, {{consolidations}}
  resolveTemplate(template: string): string {
    return template
      .replace(/\{\{input\}\}/g, this.input)
      .replace(/\{\{steps\.(\w+)\.content\}\}/g, (_, label) => 
        this.getStepOutput(label)
      )
      .replace(/\{\{consolidations\}\}/g, this.consolidationContent);
  }
}

Phase 6-8: The User Experience

The frontend brought everything together:

• Builder page with dnd-kit for drag-and-drop workflow composition
• Execution page with pipeline visualization and real-time updates
• Consolidation picker to inject project wisdom into workflows

The Plot Twist: When Prisma Gets Picky

Just when everything seemed to be working, we hit a wall. The workflow creation endpoint started throwing cryptic errors:

Unknown argument 'userId'. Available options are marked with ?.

The culprit? A subtle interaction between Prisma's type system and how we were creating workflows with nested relationships.

The Problem

Our initial approach seemed straightforward:

// ❌ This breaks Prisma's type resolution
const workflow = await prisma.workflow.create({
  data: {
    name: input.name,
    tenantId: ctx.tenantId,     // Scalar FK
    userId: ctx.user.id,        // Scalar FK
    steps: {
      create: input.steps.map(step => ({  // Nested relation create
        // ... step data
      }))
    }
  }
});

The Root Cause

Prisma uses TypeScript's conditional types to provide two different input schemas:

• WorkflowCreateInput - Uses relation objects like user: { connect: { id } }
• WorkflowUncheckedCreateInput - Uses scalar FKs like userId: string

When you include nested relation creates (steps: { create: [...] }), Prisma automatically chooses the "checked" input type where scalar foreign keys aren't valid.

The Solution

The fix required switching to Prisma's relation connect syntax:

// ✅ This works with nested creates
const workflow = await prisma.workflow.create({
  data: {
    name: input.name,
    tenant: { connect: { id: ctx.tenantId } },  // Relation connect
    user: { connect: { id: ctx.user.id } },     // Relation connect
    steps: {
      create: input.steps.map(step => ({
        // ... step data
      }))
    }
  }
});

Lessons Learned: The Developer's Compass

1. Prisma's Type System Has Opinions

When using nested relation creates, always use the { connect: { id } } pattern for foreign key references. Don't mix scalar FKs with nested relation creates—Prisma's type system won't allow it.

2. JSON Fields Need Special Handling

Use Prisma.JsonNull instead of null when resetting JSON fields. The distinction matters for Prisma's type checking.

3. TypeScript Iteration Gotchas

Use Array.from(map.entries()) instead of for...of loops on Maps to avoid --downlevelIteration TypeScript compiler errors.

4. Schema Migrations with Existing Data

When adding required columns to tables with existing data, either make them optional initially or provide sensible defaults like @default("").

The Bigger Picture

Building complex systems like workflow engines teaches us that the real challenge isn't just the business logic—it's navigating the intricate relationships between our tools. ORMs like Prisma provide incredible developer experience, but they come with their own mental models that we need to understand and respect.

The key is building systems that can evolve. Our workflow engine now supports:

• Template variable resolution for dynamic content injection
• Cost estimation before execution
• Resume and retry capabilities for robust execution
• Real-time streaming with Server-Sent Events
• Project consolidation integration for AI-powered insights

What's Next?

With the Prisma typing issues resolved, the next phase focuses on user experience refinements:

• Enhanced consolidation display showing names instead of just IDs
• Performance optimizations for large workflow executions
• Advanced template variable features
• Workflow sharing and collaboration features

The journey of building developer tools is filled with these moments—where a single type system quirk can halt progress, but solving it deepens our understanding of the tools we use every day.

Have you encountered similar Prisma type system challenges? I'd love to hear about your experiences and solutions in the comments below.