Unlocking Workflow Intelligence: How We Ensured Every Step Remembers (and Boosted Reporting!)

In the fast-paced world of software development, building intelligent systems often boils down to one fundamental principle: context. Without it, even the smartest AI or the most sophisticated workflow can flounder, repeating work or making decisions in a vacuum. Recently, we faced a critical challenge with our custom workflows: how to ensure that each step, especially review steps and downstream actions, always had full visibility into the prior work done.

This post chronicles a recent development session where we unravelled this knot, alongside delivering some significant enhancements to our reporting infrastructure.

The Quest for Context: Making Workflows Smarter

Our goal was clear: fix the workflow context passing for custom (non-template) workflows. Imagine a multi-step process where an initial analysis generates findings, but the subsequent review step or an automatic refactoring step doesn't "remember" those findings. That's precisely the problem we were solving.

The Root Cause: A Missing Link in resolvePrompt()

The core of the issue lay in how our resolvePrompt() function handled variable substitutions. It was designed to replace explicit {{steps.*}} references within a prompt template. If a template didn't explicitly ask for previous step outputs, it simply wouldn't get them. This meant custom workflows, which often start with generic prompts, and even our built-in simple review step, were operating without the rich context of prior work.

The Solution: Auto-Context Injection

Our fix introduces an elegant mechanism for "auto-context injection." We implemented a new function, buildAutoContext(ctx), within our src/server/services/workflow-engine.ts.

Here's how it works:

1. Assembling Prior Work: buildAutoContext meticulously gathers outputs from all previously completed steps.
2. Smart Summarization: To prevent context windows from overflowing, it intelligently prefers concise digests of step outputs, capping each step's contribution at 4KB.
3. Markdown Integration: All this aggregated information is then neatly packaged into a ## Previous Steps markdown block.
4. Intelligent Injection: Before resolving any prompt, resolvePrompt() now first checks if the prompt already contains explicit {{steps.*}} references. If it doesn't and there are completed steps, it automatically appends our newly built ## Previous Steps markdown block.

// src/server/services/workflow-engine.ts (simplified conceptual snippet)

function buildAutoContext(ctx: WorkflowEngineContext): string {
  let autoContext = '';
  // Logic to iterate through completed steps, prefer digests, cap size
  // and format into a markdown block like:
  // "## Previous Steps\n\n### Step 1: Initial Analysis\n[Digest of analysis output]..."
  return autoContext;
}

async function resolvePrompt(prompt: string, ctx: WorkflowEngineContext): Promise<string> {
  // ... existing variable substitution logic ...

  // Check for explicit step references
  const hasExplicitStepRefs = /\{\{steps\..*?\}\}/.test(prompt);

  // If no explicit references and there are prior steps, inject auto-context
  if (!hasExplicitStepRefs && ctx.completedSteps.length > 0) {
    const autoContext = buildAutoContext(ctx);
    prompt += `\n\n${autoContext}`; // Append the generated context
  }

  // ... final prompt resolution ...
  return prompt;
}

This ensures that whether you're using a custom workflow with a simple prompt or our built-in review step, all subsequent actions are informed by the full history of the workflow. The fix, 243cadd, is now live and making our workflows significantly more intelligent.

Beyond Workflows: Supercharging Report Generation

While fixing context passing was a major win, this session also saw substantial improvements to our reporting capabilities. We focused on persistence and flexible export options.

Persisting Reports to the Database

Gone are the days of transient reports! We've introduced a new Report Prisma model, complete with robust Row-Level Security (RLS) in prisma/schema.prisma and prisma/rls.sql. This means:

• Reports are now saved directly to the database.
• Users can only access reports associated with their projects, thanks to RLS.
• A new src/server/trpc/routers/reports.ts provides endpoints for listing, getting, and deleting reports, all filtered by project.
• Crucially, all our generateReport mutations (for autoFix, refactor, and workflows) now automatically persist the generated report to the database.

Exporting to PDF: Professional & Branded

To complement persistence, we've added a powerful PDF export feature:

• Python Powerhouse: A new scripts/md2pdf.py Python script leverages md2pdf-mermaid and Playwright with Chromium to convert markdown reports into beautiful PDFs.
• Branded Footer: Each PDF includes a branded footer, maintaining a consistent professional look.
• API Endpoint: A src/app/api/v1/reports/pdf/route.ts POST endpoint orchestrates this, calling the Python subprocess to generate the PDF on demand.
• Seamless UI Integration: Our ReportGeneratorModal and ReportsTab now feature dual download buttons, allowing users to choose between markdown and PDF formats. We also implemented a buildFilename() function to ensure consistent and user-friendly naming for downloaded files.

A minor, but necessary, update was also made to the QR code URL in src/server/services/report-generator.ts, changing it from nyxcore.app to nyxcore.cloud for consistency. The report persistence and PDF export changes were committed earlier in the session (d123ccf).

Lessons Learned: Navigating the Technical Landscape

Even with careful planning, development sessions always present opportunities for learning.

The Tricky Import Path

One small hiccup involved an incorrect import path in our new PDF API route:

// src/app/api/v1/reports/pdf/route.ts
import { auth } from '../../../middleware'; // This caused TS2307!

The Pain: TypeScript threw TS2307: Cannot find module. The Realization: The middleware.ts file was only two levels up, not three. Our route is at v1/reports/pdf/route.ts, and the middleware is at v1/middleware.ts. The Fix: A quick correction to ../../middleware resolved the issue. A classic case of counting directory levels!

Reinforcing the Value of Explicit Context

The core problem of custom workflows lacking context served as a powerful reminder: while abstraction is key, sometimes explicit context injection, even an automated one, is vital for complex, multi-step processes. Our buildAutoContext() function is a testament to finding a robust solution for a common workflow challenge.

Looking Ahead: The Road to Even Smarter Systems

With these improvements under our belt, we're already charting the next course:

1. Verify Custom Workflows: A crucial immediate next step is to thoroughly test a custom workflow (without a template) to confirm that the review step now correctly receives and utilizes prior step context.
2. RAG-based Policy Library: We're exploring the integration of a Retrieval-Augmented Generation (RAG) system for a tenant-specific policy library (DSGVO, ISO 27001, internal coding standards). User feedback strongly suggests RAG is superior to direct LLM prompting for compliance documents.
3. Context-Aware Pipelines: We'll continue advancing our "Context-Aware AutoFix & Refactor Pipelines" plan, building on the foundation laid here.
4. Setup Documentation: To ensure smooth onboarding for new team members, we'll be adding comprehensive setup instructions for the Python virtual environment and Playwright required for PDF generation.

This session was a significant leap forward in making our platform more intelligent and user-friendly. By ensuring context flows seamlessly through our workflows and providing robust reporting capabilities, we're empowering developers to build even more sophisticated solutions.