Unlocking Finer Control: Bringing Top P and Intuitive Tooltips to Our LLM Workflows

As developers, we're constantly striving to make complex systems more accessible, powerful, and user-friendly. In the world of Large Language Models (LLMs), this often means striking a delicate balance between exposing granular control and maintaining a clear, intuitive interface. This past development session was a prime example, tackling two key areas: adding a crucial new LLM parameter, Top P, and making our existing workflow settings more understandable with info tooltips.

Let's unpack the journey.

Demystifying Settings: The Power of the InfoTip

Our workflow engine allows users to configure various parameters for each LLM step, from model selection to temperature and retry mechanisms. While these settings are powerful, their purpose isn't always immediately obvious to every user. The goal was clear: provide instant, contextual help without cluttering the UI.

The Challenge: No Off-the-Shelf Solution (and Why We Built Our Own)

One of the first hurdles we encountered was the absence of a dedicated tooltip/popover component in our existing UI library. We could have pulled in a robust library like Radix Tooltip, but for a simple "hover to reveal text" functionality, that felt like overkill.

Our Solution: We opted to build a custom, lightweight InfoTip component using pure CSS for the hover popover effect. This allowed us to maintain a lean bundle size and have full control over styling and behavior.

// src/components/ui/info-tip.tsx (simplified example)
import { HelpCircle } from "lucide-react"; // Or any icon library

interface InfoTipProps {
  content: string;
  children?: React.ReactNode;
}

export const InfoTip: React.FC<InfoTipProps> = ({ content, children }) => {
  return (
    <div className="relative inline-block group">
      {children || <HelpCircle className="w-4 h-4 text-gray-400 cursor-pointer" />}
      <div className="absolute hidden group-hover:block bg-gray-800 text-white text-xs rounded-lg px-2 py-1 
                    bottom-full left-1/2 -translate-x-1/2 whitespace-nowrap z-10 
                    // Auto-positioning logic would be more complex, e.g., JS for viewport checks
                    ">
        {content}
        <div className="absolute w-2 h-2 bg-gray-800 rotate-45 -bottom-1 left-1/2 -translate-x-1/2"></div>
      </div>
    </div>
  );
};

This component was then integrated across all critical workflow step settings: Step Type, Provider, Model, Temperature, Max Tokens, Max Retries, Alternatives, Compare Providers, and Quality Gates. Now, a simple hover over the ? icon provides instant clarity, significantly improving the developer experience.

Unlocking Finer Control: Introducing Top P (Nucleus Sampling)

Beyond clarity, we wanted to empower users with more nuanced control over LLM output generation. This led us to Top P, also known as nucleus sampling. In simple terms, Top P allows you to select from the smallest possible set of words whose cumulative probability exceeds the p threshold. This provides a dynamic way to control the diversity of the output without being as rigid as Top K.

Implementing Top P end-to-end required changes across our entire stack, from the UI to the database and the LLM service adapters.

The Frontend Experience: A New Slider

On the UI side, we added a new slider for Top P (range 0.1–1.0, step 0.05) directly into the workflow step configuration, nestled between Temperature and Max Tokens.

// src/app/(dashboard)/dashboard/workflows/new/page.tsx (simplified)
// ...
<Slider
  label="Top P"
  value={step.topP}
  onValueChange={(val) => handleStepChange('topP', val)}
  min={0.1}
  max={1.0}
  step={0.05}
  infoTip="Controls the diversity of the output. Higher values consider more words, leading to more varied responses."
/>
// ...

This also involved updating our StepConfig interface to include topP: number and setting a default of 1.0 during step creation.

Backend & Database: Schema Evolution

To persist this new parameter, a database schema change was necessary. We added topP as a Float with a default of 1.0 to our WorkflowStep model in Prisma.

// prisma/schema.prisma
model WorkflowStep {
  // ... other fields
  topP        Float   @default(1.0)
  // ...
}

After modifying the schema, npm run db:push and npm run db:generate were crucial to sync the database and regenerate the Prisma client, ensuring our backend services could interact with the new field.

Our tRPC router for workflows also needed updates to incorporate topP into both the step creation and update schemas.

The LLM Service: Adapters & Conditional Logic

The core of Top P's implementation lies within our LLM service, specifically its adapters for different providers (Anthropic, OpenAI, Google, Kimi). Each provider has its own way of accepting this parameter (top_p, topP).

A key design decision here was to only send topP to providers when its value is less than 1.0. Why? Because 1.0 is typically the default behavior where all tokens are considered, and explicitly sending it might, in some edge cases, override a provider's internal default behavior or simply add unnecessary payload.

// src/server/services/llm/adapters/openai.ts (simplified)
import { LLMCompletionOptions } from '../types';

async function generateCompletion(options: LLMCompletionOptions) {
  const { prompt, temperature, maxTokens, topP } = options;

  const requestBody: any = {
    // ... other parameters
    temperature: temperature,
    max_tokens: maxTokens,
  };

  if (topP !== undefined && topP < 1.0) {
    requestBody.top_p = topP;
  }

  // ... call OpenAI API
}

Similar logic was applied to Anthropic, Google (using topP within generationConfig), and Kimi adapters, ensuring universal support across our integrated LLM providers. Finally, our central workflow-engine.ts was updated to pass the step.topP value to the LLM call.

Lessons Learned & Challenges Overcome

Every development sprint has its moments of deliberation. Here were some key takeaways:

• Top P vs. Frequency Penalty: Initially, we considered adding both Top P and Frequency Penalty. However, Frequency Penalty isn't universally supported across all LLM providers (Anthropic, for instance, lacks a direct equivalent). To avoid introducing complex, per-provider conditional logic for features that aren't broadly available, we decided to prioritize Top P due to its native support across all our integrated providers. This pragmatic approach ensures a more consistent and maintainable experience.
• Building Custom Components: The decision to create a custom InfoTip rather than importing a large library reinforced the value of lean development. For simple, specific UI needs, sometimes a few lines of well-crafted CSS and minimal JavaScript (if needed for advanced positioning) can be more efficient than a heavy dependency.
• The "Default" Dilemma: The choice to only send topP when < 1.0 is a subtle but important optimization. It respects provider defaults and avoids potential conflicts or unnecessary data transmission.

What's Next?

With InfoTips providing clarity and Top P offering fine-grained control, our LLM workflow engine is more powerful and user-friendly than ever. The immediate next steps involve thorough visual and functional QA to ensure everything renders correctly and behaves as expected. We'll be checking:

• InfoTip hover popovers across all settings.
• Top P slider appearance and functionality between Temperature and Max Tokens.

This session was a great reminder that enhancing developer experience isn't just about adding new features; it's also about making existing power more accessible and understandable.