Precision & Clarity: Enhancing LLM Workflows with Top P and Intuitive InfoTips

In the fast-evolving world of Large Language Models (LLMs), providing both powerful control and crystal-clear understanding to users is paramount. Our recent development session was squarely focused on these two pillars: enhancing user experience through insightful tooltips and empowering users with a new, granular control parameter for LLM generation: Top P.

This post will walk you through the journey of integrating these features, from frontend UI to database schema and deep into our LLM abstraction layer, highlighting the technical decisions and lessons learned along the way.

The Quest for Clarity: Introducing InfoTips

Working with LLMs often means grappling with a suite of parameters—Temperature, Max Tokens, and now Top P—each subtly influencing the model's output. For new users, or even seasoned ones needing a quick refresher, understanding what each setting does can be a barrier.

Our solution? Ubiquitous InfoTip tooltips.

Building a Lightweight InfoTip Component

Initially, we considered integrating a robust UI library component like Radix Tooltip. However, after evaluating the need for a simple hover-based popover and wanting to keep our UI bundle lean, we opted for a custom solution.

We created src/components/ui/info-tip.tsx, a reusable component designed to:

• Display a small HelpCircle icon (?).
• Show a descriptive popover on hover.
• Intelligently position itself (top or bottom) based on available viewport space, ensuring the tooltip is always visible.

// src/components/ui/info-tip.tsx (simplified structure)
import { HelpCircle } from 'lucide-react';
import React from 'react';

interface InfoTipProps {
  children: React.ReactNode;
  content: string;
}

const InfoTip: React.FC<InfoTipProps> = ({ children, content }) => {
  // Logic for positioning and CSS for hover popover
  return (
    <span className="relative group">
      <HelpCircle className="w-4 h-4 text-gray-500 cursor-pointer" />
      <div className="absolute hidden group-hover:block ...">
        {content}
      </div>
    </span>
  );
};

export default InfoTip;

This component was then seamlessly integrated across all critical workflow step settings in src/app/(dashboard)/dashboard/workflows/new/page.tsx, providing instant context for:

• Step Type, Provider, Model
• Temperature, Top P (our new parameter!), Max Tokens
• Max Retries, Alternatives, Compare Providers, Quality Gates

Now, a user simply hovers over the ? icon, and a concise explanation appears, demystifying the parameter without cluttering the interface.

Unlocking Finer Control: Introducing Top P

While Temperature controls the randomness of an LLM's output (higher = more creative/diverse), Top P (also known as nucleus sampling) offers a different, often more nuanced, way to manage output diversity. It selects the smallest set of tokens whose cumulative probability exceeds P. This means the model considers a dynamic set of high-probability tokens, leading to more coherent yet still diverse responses compared to pure temperature sampling.

Adding Top P required an end-to-end integration, touching almost every layer of our application.

The Full Stack Journey of Top P

1. User Interface (src/app/(dashboard)/dashboard/workflows/new/page.tsx):

   • A new slider was added between Temperature and Max Tokens.
   • Configured for a range of 0.1 to 1.0 with a 0.05 step, offering fine-grained control.

2. Frontend Data Model:

   • The StepConfig interface was updated to include topP: number.
   • Default values (topP: 1.0) were set in createEmptyStep() and template-based step creation to ensure backward compatibility and sensible defaults.
   • The mutation payload now explicitly includes topP: s.topP.

3. Database Schema (prisma/schema.prisma):

   • The WorkflowStep model gained a new topP field:
     prisma

     model WorkflowStep {
       // ... other fields
       topP     Float @default(1.0)
     }
     

   • After modifying the schema, npm run db:push and npm run db:generate were executed to apply the schema changes and regenerate the Prisma client, ensuring our backend could interact with the new database column.

4. Backend Services & API (src/server/trpc/routers/workflows.ts, src/server/services/workflow-engine.ts):

   • Our tRPC router for workflows was updated to validate and accept topP in both step creation and update schemas.
   • The workflow-engine.ts was modified to pass topP: step.topP directly to the underlying LLM call.

5. LLM Abstraction Layer (src/server/services/llm/types.ts):

   • The generic LLMCompletionOptions interface was extended to include topP?: number, standardizing how this parameter is handled across different LLM providers.

6. Provider Adapters (src/server/services/llm/adapters/*.ts):

   • This was a critical step. Each LLM provider has its own API for Top P.
     • anthropic.ts: Passes top_p.
     • openai.ts: Passes top_p.
     • google.ts: Passes topP within its generationConfig.
     • kimi.ts: Passes top_p.
   • Crucial Design Choice: We only send the topP parameter to the providers when its value is less than 1.0. This prevents overriding potential provider-specific default behaviors when Top P is effectively "off" or at its maximum diversity.

This comprehensive integration ensures that Top P is not just a UI element, but a fully functional parameter that influences the LLM's generation across all supported providers.

Lessons Learned & Design Choices

Every development session comes with its unique set of challenges and decisions.

The Frequency Penalty Dilemma

During the planning phase, we considered adding both Top P and Frequency Penalty. Frequency Penalty helps reduce the likelihood of the model repeating the same token multiple times. However, we quickly realized a significant hurdle: not all LLM providers natively support Frequency Penalty. Anthropic, for example, does not expose this parameter.

To maintain a consistent and simplified user experience, and to avoid complex per-provider conditional logic in our frontend and backend, we made the pragmatic decision to only implement Top P. This ensures that every workflow step, regardless of the chosen provider, can leverage the same set of core parameters without unexpected behavior or disabled features.

Lightweight UI for a Focused Experience

The choice to build a custom InfoTip component, rather than pulling in a heavier library, reflects our commitment to a performant and streamlined application. For simple UI elements, a well-crafted CSS-based solution can often be more efficient and easier to maintain.

What's Next?

With these features complete, the immediate next steps involve thorough visual QA to ensure the InfoTip popovers render correctly and Top P slider functions as expected. Once verified, these changes will be committed and pushed, bringing enhanced clarity and control to our users.

Looking ahead, we're always exploring ways to empower users further, including investigating ideas like recreating memory from git commits for project-related features, but that's a story for another blog post!

This session was a great example of how small, focused improvements in both UI/UX and core functionality can significantly enhance the developer experience and the power of our LLM-driven applications.