← nyxcore-systems
7 min read

Late Night Landing: Migrating Our AI Workflow UI and Bolstering LLM Reliability

A deep dive into a late-night dev session, covering a significant UI migration, critical LLM interaction fixes, and the successful deployment of our new 'Ipcha Mistabra' AI workflow.

LLMAIRefactoringUI/UXTypeScriptNext.jsDockerWorkflowEngineProviderSelectionProductionDeployment

It was late Friday night, the kind of late where the only sounds are the hum of your server rack and the click of your keyboard. The mission? To finally complete a major UI migration to our centralized ProviderModelPicker component and ensure our flagship AI workflow, "Ipcha Mistabra," was humming along perfectly in production.

This wasn't just about moving pixels; it was about solidifying the foundation for a more robust, extensible, and intelligent AI system.

The Mission: Centralizing Provider Selection & Unleashing 'Ipcha Mistabra'

Our primary goal for this session was twofold:

  1. Complete the ProviderModelPicker UI migration across all remaining pages. This component is crucial for allowing users (and eventually, our system itself) to intelligently select the best LLM provider and model for a given task.
  2. Deploy and monitor the 12-step "Ipcha Mistabra" workflow, a complex orchestration of AI steps designed to extract and synthesize insights from vast amounts of information.

By the time the clock ticked past midnight, both goals were not just met, but exceeded. All UI migrations were committed and deployed, and "Ipcha Mistabra" was gracefully processing at step 7/12.

The Migration Journey: Less Code, More Power

The ProviderModelPicker component represents a significant refactoring effort. Before this, various pages had their own bespoke provider/model selection logic, often leading to duplicated code, inconsistent UIs, and a headache when we wanted to add new providers or features.

The migration aimed to replace these disparate implementations with a single, reusable component. The results were satisfying:

  • src/app/(dashboard)/dashboard/auto-fix/page.tsx: Eliminated -52 lines of a custom button grid.
  • src/app/(dashboard)/dashboard/workflows/new/page.tsx: Shaved off -61 lines.
  • src/app/(dashboard)/dashboard/projects/[id]/page.tsx: The biggest win, reducing -220 lines by consolidating both blog and enrichment pickers.

This wasn't just about line counts; it was about improving maintainability, ensuring consistency, and accelerating future feature development. With commit a25f6b5 — "feat: migrate remaining pages to ProviderModelPicker component" — pushed and our production deploy successful, the UI now presents a unified front.

We even got to delete src/components/discussion/provider-picker.tsx (183 lines!) – a true sign of successful component consolidation.

Under the Hood: Bolstering AI Intelligence and Reliability

While the UI migration was a visible win, a substantial amount of work went into making our AI backend smarter and more resilient earlier in the day:

Smart Provider Recommendations

We introduced src/server/services/provider-recommendation.ts, a critical piece of infrastructure that automatically suggests the optimal provider and model based on the type of workflow step.

typescript
// Simplified example from provider-recommendation.ts
export function recommendProvider(stepType: WorkflowStepType): { provider: string; model: string } {
  switch (stepType) {
    case 'LLM_GENERATION':
      return { provider: 'google', model: 'gemini-2.5-pro' }; // Balanced for general LLM tasks
    case 'REVIEW':
      return { provider: 'anthropic', model: 'claude-sonnet-4-6' }; // Optimized for detailed review/summarization
    case 'FAST_LLM':
      return { provider: 'google', model: 'gemini-2.5-flash' }; // Cost-effective for quick, simple tasks
    default:
      return { provider: 'google', model: 'gemini-2.5-pro' };
  }
}

This service is now wired into our action-points.ts router, ensuring that both new workflows and group workflows benefit from intelligent model selection without manual intervention. This optimization saves costs, improves performance, and ensures the right tool is always used for the job.

Enrichment Engine Upgrade: More Wisdom, Deeper Insights

Our note-enrichment.ts service, responsible for providing context and depth to AI-generated content, received a significant boost. We expanded its "wisdom sources" to include:

  • Consolidations: Synthesized summaries of previous discussions.
  • Code Patterns: Relevant code snippets and best practices.
  • Workflow Insights: Learnings from past workflow executions.
  • Memory Entries: Personal knowledge base entries.

This means our AI now has a richer tapestry of information to draw from, leading to more informed and valuable outputs.

Taming Truncation: Robust LLM Output Handling

Working with large language models often means dealing with their quirks. One of the most critical fixes involved handling JSON output truncation:

The Problem: When processing long notes, our LLM call with a maxTokens of 4096 would sometimes truncate the JSON output. This led to invalid JSON, which then displayed raw, unparsed JSON in the UI and resulted in zero action points being extracted. A silent killer for workflow effectiveness!

The Solution:

  1. Increased maxTokens: Bumped the limit from 4096 to a more generous 16384. This gives the LLM more room to breathe and output complete structures.
  2. Added Regex Recovery Parser: Even with increased tokens, edge cases can occur. We implemented a regex-based recovery parser to attempt to extract valid JSON even from partially truncated or malformed LLM responses. This makes our system far more resilient.

This fix was crucial for the reliability of our AI-driven insights.

Lessons from the Trenches: What We Learned

No development session is complete without hitting a few snags. These "pain points" are often the most valuable learning opportunities.

Docker Deployment Woes & the Lifecycle Lesson

  • The Issue: During an earlier session, attempting docker compose up -d app while an old container was still running resulted in a Conflict. The container name is already in use error.
  • The Takeaway: While docker compose up -d --force-recreate or docker compose up -d --build often handles this, sometimes a clean slate is necessary. For critical deployments, understanding container lifecycles and explicitly tearing down (docker compose down app) before bringing up (docker compose up -d app) can prevent unexpected conflicts. Thankfully, this session, Recreate worked smoothly, suggesting it might have been an isolated incident.

LLM Output Reliability: Max Tokens and Recovery Parsing are Your Friends

  • The Issue: As detailed above, maxTokens limits and JSON truncation caused our AI to fail at extracting action points, displaying raw JSON instead.
  • The Takeaway: When interacting with LLMs, especially for structured outputs like JSON, always anticipate and guard against truncation.
    1. Set maxTokens Generously: If the output could be long, give the LLM ample space.
    2. Implement Robust Parsing with Fallbacks: Don't just JSON.parse(). Wrap it in try-catch, and consider regex-based recovery or other methods to salvage partial or malformed responses. Your UI (and downstream processes) will thank you.

Looking Ahead: The Road Paved

With the ProviderModelPicker migration complete and our AI backend significantly improved, the "Ipcha Mistabra" workflow (b29285b4-401b-4f50-a1d6-e739ca89b1ef) is now progressing, with step 7 of 12 running. The next steps involve monitoring its completion (the final synthesis step will pause for human review), and further enhancing our system:

  1. Workflow Completion Check: Ensure b29285b4 finishes steps 7-11, awaiting synthesis review.
  2. tRPC Procedure Cleanup: Remove discussions.availableProviders as it has zero consumers post-migration.
  3. Expand Provider Options: Consider integrating Kimi (kimi-k2-0711-preview) into our provider-recommendation.ts for review tasks.
  4. Personalized Provider Keys (BYOK): Wire userId through to resolveProvider() calls in our workflow engine and discussion service to enable users to bring their own API keys.
  5. Database Security: Add Row-Level Security (RLS) policies for the project_syncs table.

This late-night session marked a significant step forward in building a more intelligent, reliable, and user-friendly AI-powered development assistant. The satisfaction of seeing a complex migration deployed and a sophisticated workflow running smoothly in production makes those extra hours worth it. Onwards to the next challenge!

json
{
  "thingsDone": [
    "Migrated 3 remaining dashboard pages to the ProviderModelPicker component, significantly reducing UI code duplication (52, 61, and 220 lines respectively)",
    "Successfully deployed all UI migrations to production, ensuring healthy containers",
    "Implemented `provider-recommendation.ts` to auto-recommend optimal LLM providers/models per workflow step type (e.g., Gemini-Pro for LLM, Claude-Sonnet for review)",
    "Integrated auto-recommendation into workflow creation routers (`createWorkflow`, `createGroupWorkflow`)",
    "Fixed duplicate owner display for docRefs (e.g., 'SimplyLiz/SimplyLiz/CodeMCP' -> 'SimplyLiz/CodeMCP')",
    "Expanded `note-enrichment.ts` with 4 new wisdom sources: consolidations, code_patterns, workflow_insights, and memory_entries",
    "Fixed LLM JSON truncation by increasing `maxTokens` to 16384 and adding a regex recovery parser for partial JSON outputs",
    "Improved action point extraction prompt for better scanning and verification",
    "Deleted the now-obsolete `src/components/discussion/provider-picker.tsx` component (183 lines)",
    "Migrated `discussions/[id]`, `docs-pipeline`, and `refactor` pages to ProviderModelPicker in an earlier session"
  ],
  "pains": [
    "Encountered Docker container name conflict during deployment, requiring explicit `docker compose down`",
    "Experienced LLM JSON output truncation due to low `maxTokens`, leading to UI display errors and failed action point extraction"
  ],
  "successes": [
    "Achieved full UI consistency for provider/model selection across the application",
    "Significantly reduced boilerplate code through component consolidation",
    "Enhanced AI workflow intelligence with automatic and optimized provider/model selection",
    "Improved the depth and relevance of AI-generated insights via expanded enrichment sources",
    "Made LLM interactions more robust and reliable by handling output truncation gracefully",
    "Successful production deployment of a major refactor and ongoing execution of a complex AI workflow"
  ],
  "techStack": [
    "TypeScript",
    "Next.js",
    "React",
    "Docker",
    "Docker Compose",
    "LLMs (Gemini 2.5 Pro, Gemini 2.5 Flash, Claude Sonnet 4.6)",
    "tRPC",
    "Workflow Engine",
    "Production Deployment"
  ]
}