← nyxcore-systems
5 min read

Beyond the Prompt: Designing & Hardening Persona Evaluation v2 with Adversarial AI

We just wrapped up a critical design phase for Persona Evaluation v2, embracing adversarial AI for hardening and tackling a tricky LLM token limit issue. Here's how we did it.

LLMAISoftwareDesignAdversarialTestingPersonaEvaluationTroubleshootingDevOps

The journey to build robust AI systems is rarely a straight line. It's a dance between innovative design, rigorous testing, and often, unexpected challenges. This past session, we hit a major milestone in developing Persona Evaluation v2 – a system designed to thoroughly assess how our AI models interact with various user personas, including those with malicious intent.

Our goal was clear: finalize the v2 design specification, fortify it against potential exploits using our internal adversarial analysis tool, Ipcha Mistabra, and then prepare for the implementation phase.

Crafting the Blueprint: Persona Evaluation v2 Design

Designing a system for nuanced AI evaluation requires a deep dive into multiple facets. We outlined six key areas for Persona Evaluation v2, each meticulously detailed in our new design spec:

  1. PersonaProfile Interface: We're introducing 12 built-in personas (like NyxCore, Athena, Nemesis) for standard testing, alongside an auto-derivation pipeline for creating custom profiles. This ensures both breadth and flexibility.
  2. Judge Rubric Overhaul: To achieve more reliable and consistent evaluations, we're moving to a hybrid scoring model. Deterministic checks (e.g., regex, keyword matching) will form the primary evaluation layer, augmented by an LLM judge providing advisory scoring.
  3. Adversarial Persona Exploitation: This is where things get interesting. We're formalizing a Proof-of-Concept (PoC) test type, categorizing attacks into five distinct categories to proactively identify and mitigate vulnerabilities. Think of it as deliberately trying to "break" the system with crafted personas.
  4. Scoring Weight Rebalancing: Different test types have different priorities. We're implementing test-type-specific weight profiles to ensure evaluation scores accurately reflect the importance of various criteria.
  5. Auto-derivation Pipeline: Custom persona profiles won't just appear. They'll go through a human-in-the-loop approval process and be persistently stored in our database, ensuring quality and traceability.
  6. Schema Changes: To support these new capabilities, we're introducing five new nullable columns and a dedicated PersonaProfile Prisma model.

This comprehensive spec underwent an initial review by our internal code-reviewer subagent, which surfaced four critical issues that were promptly addressed. But the real test was yet to come.

Fortifying the Design: Enter Ipcha Mistabra

With the initial design polished, it was time to bring in the heavy artillery: Ipcha Mistabra. This powerful adversarial analysis workflow is designed to scrutinize our designs for potential weaknesses before a single line of production code is written. It simulates various attack vectors and probes for vulnerabilities, acting as a crucial early warning system.

Running Ipcha Mistabra against our v2 design spec was incredibly insightful. It identified:

  • 2 CRITICAL risks: Design flaws that could lead to severe exploitation.
  • 3 HIGH risks: Significant vulnerabilities requiring careful hardening.

These findings were invaluable. We immediately incorporated the necessary changes, hardening the design against these identified threats. This iterative process of design, review, adversarial analysis, and refinement is fundamental to building secure and resilient AI systems.

The Unexpected Hurdle: LLM Token Limits

Our Ipcha Mistabra workflow, like many advanced AI processes, relies on large language models (LLMs) for its analytical power. During the adversarial analysis run, we hit an unforeseen snag: truncated output.

The core of Ipcha Mistabra involves a fan-out step where multiple providers independently analyze the design. Each of these analyses was hitting the default maxTokens limit of 4096, resulting in incomplete findings. This was a critical issue, as partial adversarial insights are as good as no insights.

The Fix: We had to significantly raise the maxTokens for the relevant Ipcha steps:

  • Adversarial Analysis (fan-out): Increased from 4096 to 16384
  • Synthesis, Arbitration, Results: Increased from 4096 to 8192 (Prepare stayed at 4096 as it's less verbose)
typescript
// src/server/trpc/routers/workflows.ts
// ... inside Ipcha Mistabra workflow configuration
maxTokens: {
  adversarialAnalysis: 16384, // Crucial for comprehensive fan-out
  synthesis: 8192,
  arbitration: 8192,
  results: 8192,
  prepare: 4096, // No change
}
// ...

After implementing this change and re-running the workflow, we finally received a full, untruncated adversarial analysis, confirming the design hardening was effective. This was a sharp reminder that even with advanced tooling, managing the practical limitations of LLMs (like token context windows) remains a hands-on task.

Other Lessons Learned from the Trenches

Beyond the token crunch, we encountered a few other "classic" development hurdles:

  • The Disappearing File Act: A spec file mysteriously vanished from our working tree, despite being present in Git history. A quick git checkout HEAD -- <path> brought it back from the digital ether. Always remember your Git recovery commands!
  • Production-Only Database Access: Trying to run Prisma queries locally when the DB is production-only is a common trap. The solution was to SSH into production and query directly via docker exec nyxcore-postgres-1 psql.
  • Shell Escaping Woes with npx tsx: Running npx tsx -e "..." with commands containing special characters like ! proved problematic due to shell escaping rules interfering with esbuild syntax. The cleaner workaround was to write the script to a temporary file and then execute it via npx tsx /path/to/script.ts.

Ready for Implementation

With the design spec finalized, reviewed, and battle-hardened by Ipcha Mistabra, we're now in an excellent position to move forward. Key decisions solidified for the implementation phase include:

  • Hybrid Scoring: Deterministic checks as primary, LLM judge as advisory.
  • Human-in-the-loop: Custom persona profiles require explicit user approval.
  • DB-persisted profiles: New PersonaProfile Prisma model with draft/approved status.
  • Tiered evaluation: Quick (temp + generic jailbreaks) vs. Full (all types + tailored).
  • 12 Built-in Persona Profiles: We need to author detailed profiles for each, ensuring comprehensive test coverage.
  • New UI: A dedicated /personas/[id]/profile page for draft review, editing, and approval.

Our next immediate step is to invoke our writing-plans skill to automatically generate a detailed implementation plan from the hardened spec. This will break down the entire project into actionable tasks, paving the way for development.

This session was a testament to the iterative nature of building complex AI systems. From initial design to adversarial hardening and unexpected troubleshooting, each step brings us closer to a more robust, secure, and intelligent future.