Supercharging nyxCore: Introducing Expert Personas & Multi-Provider LLM Comparison

The world of AI is moving at lightning speed, and for nyxCore, our goal is to empower developers and users with the most flexible and intelligent tools to build incredible AI-driven workflows. This past development session was a sprint to integrate two game-changing features that push nyxCore further: Workflow Personas and Multi-Provider LLM A/B Comparison.

We're thrilled to announce that these features are now live on our development branch, ready for testing and integration. Let's dive into what they are, why they matter, and how we brought them to life.

Bringing Expertise to Your AI: Workflow Personas

Imagine not just giving an LLM a prompt, but giving it a role. What if your AI could "think" like a marketing strategist, a technical writer, or a legal expert, right from the start of a workflow? That's the power of Workflow Personas.

What it is: Workflow Personas allow you to define pre-configured "expert identities" – essentially, specialized system prompts – that can be injected into every step of your workflow. Instead of just a generic "You are a helpful AI assistant," you can now prepend your step's system prompt with something like:

## Expert: Marketing Strategist
You are a seasoned marketing strategist with 15 years of experience in digital campaigns. Your goal is to identify target audiences and craft compelling messaging that drives engagement.

## Expert: Technical Writer
You are an experienced technical writer, specializing in clear, concise, and accurate documentation for developers. Your focus is on precision and ease of understanding.

Why it matters:

• Enhanced Context: Provides a deeper, more consistent context for the LLM, leading to more relevant and higher-quality outputs.
• Domain-Specific Reasoning: Encourages the LLM to adopt a specific persona's perspective, improving the quality of reasoning for specialized tasks.
• Workflow Consistency: Ensures that a particular "mindset" is applied throughout a complex workflow, maintaining tone and focus.

How we built it: At a high level, implementing personas involved:

1. Schema Extension: Adding personaIds (an array of UUIDs) to the Workflow model in prisma/schema.prisma to link workflows to chosen personas.
2. Persona Management: Creating a new personasRouter in tRPC to handle listing and retrieving persona definitions.
3. Engine Integration: The core logic lives in src/server/services/workflow-engine.ts. A new loadPersonaSystemPrompts() function fetches the system prompts for selected personas, formats them, and prepends them to the workflow step's own system prompt before sending it to the LLM.
4. User Interface: A new multi-select persona picker was added to the workflow creation page, and persona badges now appear in the workflow settings panel for quick visibility.

The Ultimate LLM Showdown: Multi-Provider A/B Comparison

Choosing the right LLM for a specific task can be tricky. Cost, performance, quality, and even subtle stylistic differences can make or break a workflow. Our new Multi-Provider A/B Comparison feature takes the guesswork out of it.

What it is: For any given step in your nyxCore workflow, you can now select multiple LLM providers (e.g., OpenAI, Anthropic, Google, Ollama) and have the workflow engine execute that same step against all selected providers simultaneously. Their outputs are then presented side-by-side as alternatives for you to compare and choose from.

Why it matters:

• Informed Decisions: Directly compare outputs from different models using the exact same input, allowing you to pick the best performer for your needs.
• Cost Optimization: Evaluate cost-effectiveness without sacrificing quality.
• Reduced Vendor Lock-in: Easily switch between providers or use different providers for different steps based on their strengths.
• Prompt Engineering Aid: Quickly iterate on prompts by seeing how different models interpret and respond to them.

How we built it: This was a more involved change, touching several critical parts of the system:

1. Schema Extension: Added compareProviders (an array of provider names) to WorkflowStep in prisma/schema.prisma.
2. tRPC Workflow Updates: Modified stepConfigSchema and the create/update mutations for workflows and steps to handle the new compareProviders field. The estimateCost mutation now intelligently multiplies the cost based on the number of providers selected for comparison.
3. Engine Forking: The most significant change was in src/server/services/workflow-engine.ts. Inside the executeStep() function, if compareProviders has more than one entry, the engine now "forks" execution, running the step once for each specified provider. The results are then aggregated and presented as alternatives.
4. User Interface: A new multi-toggle "Compare Providers" option was added to the SortableStepCard component, allowing users to easily select which providers to compare for that specific step. This also intelligently syncs with the generateCount field. On the workflow detail page, alternative cards now display provider and model badges, and step headers show a "N providers" badge to indicate comparison mode.

Challenges & Lessons Learned

No significant feature release is without its bumps in the road. Here are a few "aha!" moments and solutions from this session:

• Type Safety vs. Zod Enums:
  • Problem: We initially tried to use a generic string[] for compareProviders in our frontend StepConfig type. However, our Zod schema on the backend was strictly z.array(z.enum(["anthropic", "openai", "google", "ollama"])). This led to TypeScript errors because string[] isn't assignable to a union literal array.
  • Lesson: When working with Zod schemas and strict types, ensure your frontend types mirror the exact literal unions defined in your schema. We resolved this by defining a literal union type (e.g., type Provider = "anthropic" | "openai" | "google";) and using Provider[] in our frontend types. This enforces strict type checking from end to end.
• Missing Default Values:
  • Problem: When adding compareProviders to stepConfigSchema, we forgot to include it in the default step configuration for new workflows. This caused TS2741 errors about missing properties.
  • Lesson: Always remember to update default values for new fields, especially when they are non-optional or have specific default requirements, to maintain type completeness and prevent runtime errors. Adding compareProviders: [] to the default stepConfig fixed this.
• Dev Server Shenanigans:
  • Problem: During rapid development, I attempted to run two nyxCore dev servers simultaneously (an older branch and the new one) on the same port, leading to unpredictable caching issues and style conflicts.
  • Lesson: A clean slate is often the fastest path forward. When encountering strange behavior, especially with frontend frameworks, kill all processes on the relevant port (lsof -ti:3000 | xargs kill -9), clear your .next cache, and perform a fresh restart. This ensures you're working with a completely rebuilt and consistent environment.

The Journey to Done

This session culminated in a robust set of changes across the entire stack:

• Database Schema: Updated prisma/schema.prisma to support Workflow.personaIds and WorkflowStep.compareProviders.
• Backend Logic: New tRPC router for personas, significant updates to workflows.ts for handling new fields in mutations, and workflow-engine.ts for persona prompt injection and multi-provider execution forking.
• Frontend UI: Integrated new persona pickers, multi-provider toggles, and display badges in the dashboard.
• Type Safety: Updated constants.ts and various component types to reflect the new data structures.

The entire feature set is now committed as aa37799 on main, with the dev server running clean and ready for action.

What's Next?

With these foundational pieces in place, our immediate next steps involve rigorous testing:

1. UI Verification: Ensuring all styles render correctly after the cache clear.
2. Data Seeding: Adding test personas to the database for comprehensive manual testing.
3. Functional Testing:
   • Creating a workflow with multiple personas and verifying their system prompts in the output.
   • Setting up a step with compareProviders for different LLMs and confirming side-by-side alternatives.
   • Testing alternative selection and workflow continuation.
   • Verifying backward compatibility for generateCount (temperature variations) without compareProviders.
   • Checking that personaIds and compareProviders are correctly preserved during workflow duplication.

These new features significantly enhance nyxCore's capabilities, offering unprecedented control and insight into your AI workflows. We're excited to see how they empower our users to build even more sophisticated and intelligent applications!