← nyxcore-systems
9 min read

From Rigid to Reactive: Building a Dynamic Workflow Engine with LLMs

We just transformed our rigid, hardcoded workflow system into a fully dynamic, LLM-powered builder and execution engine, unlocking unprecedented flexibility and control for our users.

workflow-engineLLMPrismatRPCNext.jstypescriptsoftware-architecturefrontendbackenddeveloper-experience

Every developer eventually faces the limitations of a hardcoded system. For us, it was a 5-step workflow that, while functional, became a straitjacket for innovation. Our users needed more. They needed to define their own multi-step processes, chain AI outputs, pause for human review, and retry failed steps. In short, they needed a dynamic workflow builder and execution engine.

And after an intense development session, I'm thrilled to report: we built it.

This isn't just about adding new features; it's a complete architectural overhaul, touching every layer from the database schema to the UI. We've replaced rigidity with reactive flexibility, empowering users to orchestrate complex LLM interactions with ease.

Let's dive into how we pulled it off.

The "Why": Breaking Free from Hardcoded Chains

Our original system was simple: five predefined steps, executed sequentially. If a user wanted a different number of steps, or a custom prompt, or a review gate in the middle, they were out of luck. This meant every new use case required code changes, deployments, and a slow feedback loop.

Our goal was ambitious:

  1. Dynamic Step Definition: Users define any number of steps, each with custom configurations.
  2. Output Chaining: The output of one step seamlessly feeds into the prompt of another.
  3. Flexible Execution: Pause for human review, intelligent retries, and real-time progress.
  4. Template-Driven: Ability to save and share workflow templates.

This vision required a fundamental shift.

Laying the Foundation: Schema & Core Services

The first domino to fall was the database schema. Our existing WorkflowStep model was too simplistic.

Schema Evolution: Unlocking Flexibility (Phase 1)

We supercharged WorkflowStep with 13 new fields, including label, prompt, systemPrompt, provider, model, temperature, maxTokens, stepType, enabled, retryCount, maxRetries, durationMs, tokenUsage, costEstimate, and errorMessage. This allowed each step to be a fully configurable LLM interaction.

Crucially, we introduced WorkflowTemplate to store reusable configurations and added userId and description to the Workflow model itself for better user management and context. We also dropped a @@unique constraint on WorkflowStep to allow more flexible step naming within a workflow.

Shared Provider Resolution: Clean LLM Integration (Phase 2)

As we integrate more LLM providers and support "Bring Your Own Key" (BYOK) functionality, a centralized resolveProvider(providerName, tenantId) function became essential. We refactored this out of our discussion-service into a shared src/server/services/llm/resolve-provider.ts module. This promotes code reuse and ensures consistent LLM provider instantiation across the application.

Defining the Building Blocks: Constants (Phase 3)

To manage the various types of steps and built-in templates, we created WORKFLOW_STEP_TYPES and a StepTemplate interface. We then populated STEP_TEMPLATES with 16 entries – 7 basic LLM interaction types and 9 for our complex "Deep Build Pipeline" demo (more on that later). This provides a single source of truth for step definitions and allows our UI to dynamically render options.

The Brains of the Operation: The Workflow Engine (Phase 4)

This is where the magic happens. The src/server/services/workflow-engine.ts file grew to nearly 400 lines, becoming the orchestrator of all workflow execution.

Context Management & Prompt Resolution

At the heart of the engine is ChainContext, which holds stepOutputs and stepLabels. This context is crucial for:

  • resolvePrompt(): Our templating engine. It intelligently resolves variables in prompts, allowing steps to chain outputs. For instance, {{input}}, {{input.field}}, and {{steps.Label.content}} or {{steps.Label.field}} allow dynamic content injection from previous steps or initial workflow input. This is what truly enables dynamic chaining.
  • buildChainContext(): Rebuilds the context from the database, enabling seamless resume-after-pause functionality.

runWorkflow(): An AsyncGenerator for Real-Time Streaming

The runWorkflow() function is an AsyncGenerator, designed for streaming real-time updates to the client. This is key for providing a live, engaging user experience during execution. It handles:

  • Resume Logic: Skips already completed steps if resuming a paused workflow.
  • Disabled Steps: Skips steps marked as disabled by the user.
  • Review Gates: Pauses execution at designated "review" steps, allowing human intervention (unless "YOLO mode" is enabled!).
  • Retries with Backoff: Automatically retries failed steps with exponential backoff up to a maxRetries limit.
  • Event Streaming: Emits progress | text | done | error | retry | skipped | paused events, providing granular updates including stepId, stepLabel, tokenUsage, costEstimate, and durationMs.

Pre-Execution Cost Estimation

Before running a workflow, users can get an estimateWorkflowCost() using COST_RATES. This transparency helps users manage their LLM spend.

Opening the Gates: API & UI (Phases 5-8)

With the engine built, we needed to expose its power through a robust API and an intuitive user interface.

tRPC Router: A Type-Safe API Surface (Phase 5)

Our src/server/trpc/routers/workflows.ts router expanded significantly to manage all workflow operations. We created dedicated sub-routers for steps (add, update, remove, reorder, retry, overrideOutput) and templates (list, create). The main router now supports: list, get, create (from scratch or template), update, start, resume, pause, duplicate, estimateCost, and delete.

Crucially, we implemented LLM rate limiting on start, resume, and steps.retry endpoints to prevent abuse and manage API quotas.

The Workflow Builder: Drag, Drop, Configure (Phase 6)

The src/app/(dashboard)/dashboard/workflows/new/page.tsx is an entirely new page dedicated to building workflows. It features:

  • Intuitive Inputs: Name, description, and a "YOLO mode" toggle (for skipping review steps).
  • Template Picker: Start from one of our 3 built-in templates (including the Deep Build Pipeline).
  • Sortable Step List: Powered by @dnd-kit/sortable with GripVertical drag handles, allowing users to easily reorder steps.
  • Collapsible Step Cards: Each card provides extensive configuration options: step type, LLM provider, model, temperature, max tokens, prompt, system prompt, enable toggle, and delete.
  • Quick-Add Buttons: For rapidly adding common step types.
  • Sticky Submit Footer: Ensures the save button is always accessible, even on mobile.

The Execution Page: Live Feedback & Interaction (Phase 7)

The src/app/(dashboard)/dashboard/workflows/[id]/page.tsx was completely rewritten to provide a rich, interactive execution experience:

  • Vertical Pipeline Visualization: Status dots and connecting lines visually represent the workflow's progress.
  • Live SSE Streaming: Accumulates text output per step in real-time, thanks to our AsyncGenerator engine and SSE endpoint (/api/v1/events/workflows/[id]).
  • Markdown Rendering: Completed step outputs are beautifully rendered.
  • Inline Output Editing: For "review" steps (or any step post-completion), users can inline edit the output, and subsequent steps will chain the edited content.
  • Retry Button: On failed or completed steps, allowing users to re-run specific parts of the workflow.
  • Settings Panel: Access to YOLO toggle, cost estimate, clone, and delete actions.
  • Context-Sensitive Actions: "Run," "Pause," and "Resume" buttons adapt based on the workflow's current status.
  • Next.js 15 Async Params: Leveraged for efficient data fetching.

List Page Update (Phase 8)

A minor but important update to src/app/(dashboard)/dashboard/workflows/page.tsx, adding a prominent "New" button, displaying descriptions and step counts, and showing step labels in tooltips.

A Real-World Demo: The Deep Build Pipeline

To demonstrate the power of this new system, we created a 9-step Deep Build Pipeline template:

  1. Idea Generation
  2. Research & Context Gathering
  3. Feature Brainstorm
  4. Review (Human Gate 1)
  5. Extend & Improve
  6. Review (Human Gate 2)
  7. Project Wisdom Extraction
  8. Final Improvements
  9. Implementation Prompts

Each step has tailored system prompts and dynamically chains outputs using {{steps.Label.content}}. The two "Review" gates pause the workflow for human approval (unless YOLO mode is active). The final step generates self-contained AI coding prompts, ordered by dependency, ready for development. This template truly showcases the power and flexibility we've built.

Lessons Learned: Navigating the Development Minefield

No major project comes without its challenges. Here are a few "pains" that turned into valuable lessons:

  • Prisma Required Fields on Existing Data:

    • Problem: Adding required fields (label, prompt) to WorkflowStep (which had existing rows) caused npx prisma db push to fail: "Added the required column without a default value. There are N rows."
    • Solution: Temporarily added @default("") to the new required fields and made existing required fields like userId optional (String? and User? relation). After db push, we ran a one-off backfill script using npx tsx -e to populate the existing rows with meaningful data. Then, we could remove the @default("") if desired (or keep it if an empty string is an acceptable default).
    • Takeaway: Always anticipate schema changes affecting existing data. Plan for defaults or backfills before running migrations/pushes.

  • Prisma Json Field null vs. Prisma.JsonNull:

    • Problem: Trying to set a Json field in Prisma to null (e.g., workflowStep.update({ data: { config: null } })) resulted in a TypeScript error: Type 'null' is not assignable to type 'NullableJsonNullValueInput | InputJsonValue'.
    • Solution: Prisma has a specific Prisma.JsonNull enum for explicitly setting JSON fields to null.
    • Takeaway: Be aware of Prisma's specific types for handling Json fields, especially when trying to set them to null.

  • TypeScript Map Iteration without --downlevelIteration:

    • Problem: Iterating directly over a Map using for (const [k, v] of map) caused a TypeScript error ("can only be iterated through when using '--downlevelIteration'") because our tsconfig didn't have that flag.
    • Solution: The workaround was to convert the Map to an array first: Array.from(map.entries()).
    • Takeaway: Be mindful of tsconfig settings and their impact on common language features, especially when working with older JS targets or specific library requirements. Array.from() is a reliable fallback.

What's Next?

The core engine and UI are functional, but the journey continues. Our immediate next steps involve rigorous testing:

  • Verify the full flow: template creation, step configuration, execution, and SSE streaming.
  • Confirm output chaining ({{steps.Label.content}}) works flawlessly.
  • Test the review step pause/resume in non-YOLO mode.
  • Validate retry functionality on failed steps and its impact on subsequent steps.
  • Ensure inline output editing correctly influences chained content.
  • Test drag-to-reorder persistence in the builder.
  • Verify deep duplication of workflows.

This project was a massive undertaking, but seeing the dynamic builder and live execution in action makes every line of code worth it. We've truly empowered our users to build and execute powerful, custom LLM workflows, opening up a world of possibilities.

What dynamic workflows are you building? Share your thoughts in the comments!

json
{
  "thingsDone": [
    "Implemented dynamic workflow builder UI",
    "Developed a robust workflow execution engine with AsyncGenerator",
    "Designed flexible database schema for steps and templates",
    "Created real-time SSE streaming for workflow progress",
    "Integrated LLM provider resolution and cost estimation",
    "Built a comprehensive tRPC API for workflow management",
    "Added features like output chaining, review gates, and intelligent retries",
    "Developed a demo 'Deep Build Pipeline' template"
  ],
  "pains": [
    "Prisma schema migration with existing data and new required fields",
    "Handling `null` values for Prisma Json fields",
    "TypeScript Map iteration compatibility without `--downlevelIteration`"
  ],
  "successes": [
    "Achieved full dynamic workflow functionality",
    "Seamless real-time user experience with SSE",
    "Modular and reusable LLM provider service",
    "Intuitive drag-and-drop workflow builder",
    "Robust error handling and retry mechanisms",
    "Demonstrated complex chaining with 'Deep Build Pipeline'"
  ],
  "techStack": [
    "Next.js 15",
    "TypeScript",
    "Prisma",
    "PostgreSQL",
    "tRPC",
    "@dnd-kit/sortable",
    "Server-Sent Events (SSE)",
    "LLM APIs (via custom provider layer)"
  ]
}