← nyxcore-systems
5 min read

Taming AI Hallucinations: How We Turned a Critical Bug into a Feature for Smarter Workflows

We faced a critical bug where our AI workflow engine started hallucinating irrelevant prompts. This post dives into finding the root cause, implementing systemic fixes, and how we're now turning that 'bug' into an exciting new feature: controlled AI ideation.

AIDebuggingWorkflowEngineTypeScriptLLMDevelopmentFrontendBackend

Every developer knows the thrill of squashing a critical bug, especially one that throws a wrench into the core functionality of your application. Recently, we tackled a particularly tricky one: our AI-powered workflow engine started hallucinating. Instead of generating task-specific prompts, it was conjuring up scenarios about healthcare patient onboarding and e-commerce checkout flows, completely unrelated to the actual workflow. This wasn't just a minor glitch; it was a BLOCKER.

This post chronicles our journey from perplexed debugging to a robust solution, and even how we're now planning to leverage this "bug" as an intentional feature for creative ideation.

The Mystery of the Misguided Prompts

Our workflow engine is designed to guide various AI personas (like "Sasha Lindqvist/Engineering" or "Noor Okafor/Security") through complex tasks, generating context-aware prompts at each step. The problem? One specific workflow, "nyxCore - Hetzner Deployment," was consistently producing entirely generic, off-topic content in its second step, "Generate Code Prompts."

Imagine expecting code generation prompts and instead getting this:

"As a healthcare professional, collaborate with the e-commerce specialist to design a patient-centric online booking system, considering payment gateways and data privacy regulations."

While interesting, it was wildly incorrect for a "Hetzner Deployment" workflow.

Unearthing the Root Cause

Our investigation led us down a rabbit hole into the workflow's configuration. The "Generate Code Prompts" step was referencing {{steps.Extract Features.content}}. A quick check revealed the truth: there was no step with the label "Extract Features" in that workflow.

This was our "Aha!" moment. When the workflow engine tried to resolve {{steps.Extract Features.content}}, it found nothing. An empty or unresolved input to a Large Language Model (LLM) is like giving a blank canvas to an artist – it will try to fill the void based on the broadest context it has, which, in our case, was the general specializations of the personas involved (e.g., "Engineering," "Security," "DSGVO"). The LLM was essentially improvising, leading to the "hallucinated" healthcare and e-commerce scenarios.

Engineering the Fix: Systemic Solutions

Identifying the root cause was just the first step. We needed to implement robust, systemic fixes to prevent this from happening again and to handle existing broken workflows gracefully.

1. Smarter Prompt Resolution in the Workflow Engine

The core of the problem lay in how src/server/services/workflow-engine.ts handled missing step references.

  • detectMissingStepRefs(): We introduced a new helper function to explicitly scan for {{steps.X.content}} patterns where X doesn't correspond to an existing step label.
  • resolvePrompt() Enhancements:
    • Instead of silently returning an empty string for broken references, we now return a clear warning: [WARNING: Step "X" not found. Available steps: ...]. This makes debugging much easier.
    • Crucially, if a step's prompt does contain broken references, we now implement a fallback mechanism: the resolvePrompt() function will automatically inject the outputs of all previous steps into the LLM's context. This ensures the LLM always has relevant information, even if a specific reference is broken, mitigating hallucination.
    • We also added warning yields before execution paths, emitting progress events with details about any missing references.

2. Preventing Future Breaches: Workflow Validation

To catch these issues before they even hit the workflow engine, we implemented validation at the API level in src/server/trpc/routers/workflows.ts.

  • create Mutation Validation: Before a new workflow can be created, its step prompts are now scanned for {{steps.X.content}} references. If any X doesn't match an existing step label, the API throws a TRPCError BAD_REQUEST with detailed information. This is a crucial line of defense, ensuring data integrity from the start.

Lessons Learned: Navigating the Development Trenches

No debugging session is complete without its share of minor frustrations and valuable insights.

  • Prisma Query Scaffolding: For non-trivial Prisma queries, especially those involving complex filters or updates, using npx tsx -e '...' directly in the shell proved cumbersome due to shell escaping issues with special characters. The lesson: always create a temporary .ts script file (e.g., scripts/temp-query.ts), run it with npx tsx scripts/temp-query.ts, and then delete it. It saves a lot of headaches.
  • Schema Detective Work: We hit a couple of snags trying to select fields like templateId on Workflow (it's on WorkflowTemplate) and compareOutputs on WorkflowStep (it's alternatives for JSON output and checkpoint for review data). This reinforced the importance of thoroughly understanding your Prisma schema and model definitions, especially when working with polymorphic or evolving data structures.

The Silver Lining: Controlled Hallucination

Perhaps the most exciting outcome of this whole ordeal is the realization that the "bug" itself, when controlled, can be a powerful feature.

The LLM's tendency to "dream up" scenarios when given an empty prompt, based on the personas' specializations, sparked an idea: What if we could intentionally trigger this behavior for creative ideation?

This led to the concept of /init-dream (or a similar feature).

  • Concept: Give a set of personas a vague or even empty prompt intentionally, perhaps with some domain hints, and let them generate creative, collaborative scenarios.
  • Use Case: Brainstorming, discovering unexpected interactions between personas, or exploring novel solutions to ill-defined problems.
  • Parameters: We envision parameters for persona sets, optional domain hints, creativity temperature, and desired output formats.

This "controlled hallucination" could manifest as a new workflow step type (dream), a dedicated feature, or a special workflow template. The key insight is turning a system's unexpected behavior into a valuable tool for innovation.

What's Next?

With the critical bug resolved, our immediate next steps include:

  • Designing /init-dream: Deep-diving into the feature design to make controlled hallucination a reality.
  • Expanding Validation: Considering adding step cross-reference validation to the updateStep mutation as well, not just create.
  • UI Enhancements: Implementing a UI indicator in the workflow builder to visually warn users about broken step references.
  • Workflow Cleanup: Informing users that existing workflows with broken references (like our problematic 5d30703a-... example) will now benefit from the auto-context fallback, and they might want to re-run them.

This session was a stark reminder that even critical bugs can lead to profound insights and exciting new features. By understanding why our AI was hallucinating, we not only fixed the problem but also discovered a powerful new avenue for creative exploration. Onwards to smarter, and sometimes, more imaginatively "dreaming" workflows!