Building an AI-Powered Code Review Assistant: Key Points Extraction and Smart Workflow Management

Late evening coding sessions often produce the most interesting breakthroughs. Last night was one of those nights where everything clicked into place—we successfully implemented an AI-powered system that can extract actionable insights from workflow reviews and turn them into interactive, manageable tasks.

The Vision: From Review Chaos to Actionable Insights

Anyone who's worked with AI-assisted code reviews knows the problem: you get back walls of text with valuable insights buried in verbose explanations. Our goal was to build a system that could automatically extract the key actionable points from these reviews and present them in a way that developers could actually use.

Here's what we built:

Phase 1: Smart Key Points Extraction

The heart of the system is a new service that leverages Claude Haiku for fast, structured extraction of review insights:

// src/server/services/review-key-points.ts
export async function extractKeyPoints(reviewOutput: string): Promise<ReviewKeyPoint[]> {
  // Uses Claude Haiku for fast JSON extraction
  // Validates structure, assigns UUIDs, enforces limits
  // Caps at 50 items with 200/2000 char field limits
}

We chose Claude Haiku specifically for this task—it's fast enough for real-time extraction while being reliable for structured JSON output. The service automatically validates the response shape, assigns unique IDs to each point, and enforces reasonable limits to prevent overwhelming the user.

Phase 2: Workflow Engine Integration

The magic happens in the workflow engine. After any review step completes successfully, the system automatically extracts key points and stores them alongside the checkpoint data:

// Integrated into workflow-engine.ts
if (reviewStep.success) {
  const keyPoints = await extractKeyPoints(reviewStep.output);
  checkpoint.keyPoints = keyPoints;
}

This seamless integration means developers get structured insights without any additional manual work.

Phase 3: Interactive Management Interface

We built a comprehensive React component that turns these extracted points into an interactive experience:

// Components include:
// - Severity summary bar with visual indicators
// - Grouped key points with inline editing
// - Per-item actions (keep/edit/discard)
// - Bulk operations for efficient management

The interface provides both granular control (edit individual points) and bulk operations (accept all, recreate with hints, or start fresh). Users can make inline edits to refine suggestions or discard points that aren't relevant.

Phase 4: Smart Workflow Recreation

Perhaps the most powerful feature is the ability to recreate workflow steps with AI-generated hints. When users select key points and choose "Recreate with Hints," the system:

1. Resets the target step and all subsequent steps
2. Injects selected key points as structured hints into the step prompt
3. Prevents hint accumulation through smart prompt parsing

Lessons Learned: The Technical Challenges

Every development session has its pain points. Here are the key challenges we overcame:

TypeScript and Prisma JSON Fields

Challenge: Storing complex nested objects in Prisma's JSON fields led to type assignment errors.

// This failed:
checkpoint.keyPoints = extractedPoints; // TS2322 error

// Solution: Cast through unknown (standard pattern)
checkpoint.keyPoints = extractedPoints as unknown as Prisma.InputJsonValue;

Preventing Hint Accumulation

Challenge: Multiple recreations would infinitely append hints to prompts.

Solution: Parse and strip previous hints before adding new ones:

const cleanPrompt = prompt.split(HINT_SEPARATOR)[0];
const newPrompt = cleanPrompt + HINT_SEPARATOR + newHints;

Data Preservation During Updates

Challenge: Simple object assignment was destroying existing checkpoint data.

Solution: Use spread syntax to merge rather than overwrite:

// Wrong: destroys keyPoints
checkpoint = { reviewNotes };

// Right: preserves existing data
checkpoint = { ...existingCheckpoint, reviewNotes };

Enhanced Workflow Metadata

As a bonus, we enhanced the workflow list page with rich metadata aggregation. Each workflow card now displays:

• Total cost and token usage
• Step progress (completed/total)
• Execution duration
• Creation timestamp

This gives developers immediate insight into workflow performance and status without diving into individual workflows.

The Results

The system is now live and working beautifully. Here's what we achieved:

• ✅ Automatic extraction of actionable insights from AI reviews
• ✅ Interactive management interface with inline editing
• ✅ Smart workflow recreation with hint injection
• ✅ Enhanced workflow metadata for better overview
• ✅ Type-safe implementation with proper error handling

Next Steps

While the core functionality is complete, there are several areas for future enhancement:

1. Shared Type Definitions: Currently, ReviewKeyPoint types are duplicated between server and client code
2. Security Hardening: Adding rehype-sanitize for defense against potential XSS in LLM output
3. Component Refactoring: The main workflow page is approaching 1,640 lines—time to extract some components

Technical Stack

• AI Model: Claude Haiku for fast structured extraction
• Backend: TypeScript with tRPC for type-safe APIs
• Database: Prisma with JSON field storage
• Frontend: React with inline editing capabilities
• Validation: Zod schemas for runtime type checking

Conclusion

Building AI-powered developer tools requires careful attention to both the AI integration and the human interface. The key insight from this project was that raw AI output, no matter how good, needs to be structured and made actionable to be truly useful.

The combination of fast AI extraction, persistent storage, and interactive management creates a workflow that feels natural and powerful. Developers can now get the benefits of AI review insights without drowning in unstructured feedback.

Sometimes the best coding sessions happen late at night when everything just clicks into place. This was one of those nights.

The complete implementation is available in commit 9172e61 with full TypeScript coverage and comprehensive error handling.