Taming LLM Enrichment & Action Points: Lessons from a Feature Sprint

Every feature sprint brings its unique set of challenges and triumphs. Recently, our team tackled a significant upgrade to how users interact with AI-powered note enrichment and how action points are managed within our project detail pages. The goal was twofold: give users granular control over their AI's "wisdom," and bring much-needed order to the action points generated from their notes.

This post will walk you through the journey, from initial design decisions to the nitty-gritty code changes and, most importantly, the hard-won lessons learned along the way.

The Mission: Smarter AI, Organized Actions

Our main objectives for this sprint were clear:

1. Empower Users with LLM Controls: Allow users to choose specific LLM providers, override models (e.g., from GPT-4 to Claude 3), and select a "persona" for note enrichment directly on a per-note basis.
2. Group Action Points by Source: When our AI generates action points from a note, we wanted to clearly group these action points under their original source note on the project's "Actions" tab. This makes traceability and context much easier.

We're happy to report that these features are now live, committed, and pushed! Let's break down how we got there.

Engineering the Enrichment Controls

The core of the LLM control feature revolved around extending our existing enrichNoteWithWisdom() service. Previously, it was a fairly rigid function. Now, it accepts three crucial parameters: providerName, modelOverride, and personaId.

Tightening the API Contract with Zod Enums

One of the early decisions involved how we'd validate the providerName coming from the frontend. Initially, I reached for z.string() in our tRPC procedure:

// Initial (problematic) approach
export const enrich = t.procedure
  .input(
    z.object({
      noteId: z.string().uuid(),
      provider: z.string(), // Uh oh, this is too broad!
      model: z.string().optional(),
      personaId: z.string().uuid().optional(),
    })
  )
  // ...

Lesson Learned: Don't trust arbitrary strings for enum-like values. A quick code review (and a moment of self-reflection) highlighted a critical flaw: z.string() allows any string. This could lead to invalid providers being passed to our backend, causing runtime errors or even potential security vulnerabilities if not handled meticulously.

The fix was straightforward but essential: leverage Zod's enum capabilities, tied directly to our backend's LLM_PROVIDERS constant.

// Corrected approach using z.enum
import { LLM_PROVIDERS, LLMProviderName } from 'src/shared/types'; // Assuming shared types

export const enrich = t.procedure
  .input(
    z.object({
      noteId: z.string().uuid(),
      provider: z.enum(LLM_PROVIDERS), // Much better!
      model: z.string().optional(),
      personaId: z.string().uuid().optional(),
    })
  )
  // ...

This not only provides robust validation at the API boundary but also gives us strong type safety (LLMProviderName) throughout our application. It's a small change, but it significantly improves the reliability and security of our API.

Guarding Against the Unknown

Another important addition was a guard for missing LLM provider entries, especially relevant for local setups or new providers like Ollama. If a configured model isn't found in our FAST_MODELS mapping, we now throw a clear, descriptive error instead of silently failing or returning unexpected results. This makes debugging and system maintenance much easier.

Bringing Order: Grouping Action Points

To group action points by their source note, we needed a fundamental change to our database schema.

Database Schema Evolution

We added a sourceNoteId foreign key to our ActionPoint model in prisma/schema.prisma:

// prisma/schema.prisma
model ActionPoint {
  id           String     @id @default(uuid())
  content      String
  isCompleted  Boolean    @default(false)
  createdAt    DateTime   @default(now())
  updatedAt    DateTime   @updatedAt
  projectId    String
  project      Project    @relation(fields: [projectId], references: [id], onDelete: Cascade)
  tenantId     String
  tenant       Tenant     @relation(fields: [tenantId], references: [id], onDelete: Cascade)

  // New FK to link to the source note
  sourceNoteId String?    @map("source_note_id")
  sourceNote   ProjectNote? @relation("SourceNoteActionPoints", fields: [sourceNoteId], references: [id])

  @@index([projectId])
  @@index([tenantId])
}

model ProjectNote {
  id           String        @id @default(uuid())
  title        String?
  content      String
  projectId    String
  project      Project       @relation(fields: [projectId], references: [id], onDelete: Cascade)
  tenantId     String
  tenant       Tenant        @relation(fields: [tenantId], references: [id], onDelete: Cascade)
  createdAt    DateTime      @default(now())
  updatedAt    DateTime      @updatedAt

  // New back-relation to easily fetch action points for a note
  actionPoints ActionPoint[] @relation("SourceNoteActionPoints")

  @@index([projectId])
  @@index([tenantId])
}

This simple addition, along with the actionPoints ProjectNote[] back-relation on ProjectNote, allows us to easily fetch and display action points grouped by their originating note.

The ActionPointGroups Component

On the frontend, we built a new ActionPointGroups React component. This component takes the list of action points, groups them by sourceNoteId, and renders them under collapsible headers that display the source note's content. To ensure performance, especially with many notes and action points, we used useMemo for the grouping logic, preventing unnecessary re-calculations on re-renders.

// Simplified example of grouping logic within ActionPointGroups
const groupedActionPoints = useMemo(() => {
  const groups = new Map<string, ActionPoint[]>();
  actionPoints.forEach(ap => {
    const noteId = ap.sourceNoteId || 'ungrouped'; // Handle action points without a source note
    if (!groups.has(noteId)) {
      groups.set(noteId, []);
    }
    groups.get(noteId)?.push(ap);
  });
  return Array.from(groups.entries());
}, [actionPoints]);

// ... render logic iterating over groupedActionPoints

The "Pain Log" to "Lessons Learned" Journey

No sprint is without its hiccups. Here's how we navigated some common pitfalls:

1. The Per-Note State Dilemma

The Problem: We initially tried to implement a shared state for enrichment settings across all notes on the page. The idea was simple: one set of controls affects the enrichment of whichever note you're looking at.

The Failure: Code review quickly pointed out the flaw: changing settings for one note would inadvertently change the settings for all notes. Not exactly what our users wanted for granular control!

The Lesson: For UI elements that need specific, isolated configurations, per-item state is often the robust solution. We switched to a Map<noteId, EnrichSettings> to store distinct settings for each note. This allowed us to getEnrichSettings() and patchEnrichSettings() for a specific note without affecting others. It's a classic example of when global state can become a liability.

2. Prisma's update and Ownership Checks

The Problem: When updating an ActionPoint (e.g., marking it complete), we wanted to ensure the user truly owned that action point and its parent project/tenant. My initial thought was to use a compound where clause in Prisma's update method:

// Attempted (and failed) Prisma update
await prisma.actionPoint.update({
  where: {
    id: actionPointId,
    tenantId: ctx.session.user.tenantId, // This doesn't work!
    projectId: projectId, // Nor this!
  },
  data: { isCompleted: true },
});

The Failure: Prisma's update operation's where clause is quite specific; it primarily expects a unique identifier (@id or @@unique). It doesn't support arbitrary compound AND conditions across non-unique fields in the where directly for update.

The Lesson: Always perform explicit ownership checks before sensitive update or delete operations. The workaround was to first findFirst with the compound where to verify ownership and existence, and then proceed with the update using only the unique id.

// Corrected approach: Explicit ownership check
const existingActionPoint = await prisma.actionPoint.findFirst({
  where: {
    id: actionPointId,
    tenantId: ctx.session.user.tenantId,
    projectId: projectId,
  },
});

if (!existingActionPoint) {
  throw new TRPCError({ code: 'NOT_FOUND', message: 'Action point not found or unauthorized.' });
}

await prisma.actionPoint.update({
  where: { id: actionPointId }, // Now we can use the simple ID
  data: { isCompleted: true },
});

This pattern, while adding an extra database call, significantly enhances security and ensures data integrity by preventing unauthorized modifications.

3. Persona Query Scoping

A subtle but critical security fix involved scoping persona queries. We ensured that when a user requests a persona, it's either a built-in system persona or one explicitly owned by their tenant. This prevents users from accidentally (or maliciously) accessing personas defined by other tenants.

Looking Ahead

With these features shipped, our next steps involve:

• End-to-End Verification: A thorough run-through to ensure everything flows smoothly from configuring enrichment to seeing action points grouped correctly.
• Refactoring: The NoteEnrichmentPanel in our page file is growing. We'll consider extracting it into a standalone component to keep our codebase clean and maintainable.
• Local Dev Environment: Noticing the .claude/ directory is untracked – a small reminder that developer-specific configs are best left out of source control.

This sprint was a great reminder that even seemingly straightforward features can uncover interesting architectural and security considerations. By embracing these challenges and documenting our lessons, we continue to build a more robust and user-friendly product.

What are your go-to patterns for managing per-item state or enforcing ownership in your applications? Share your thoughts in the comments!

{
  "thingsDone": [
    "Added provider/model/persona chooser for note enrichment",
    "Implemented grouping of action points by source note in project detail",
    "Extended `enrichNoteWithWisdom()` to accept new LLM parameters",
    "Added `sourceNoteId` FK and `sourceNote` relation to `ActionPoint` in Prisma schema",
    "Implemented `actionPoints` back-relation on `ProjectNote`",
    "Built per-note enrichment settings UI with `Map<noteId, EnrichSettings>` state",
    "Built `ActionPointGroups` component with collapsible headers and memoized grouping",
    "Fixed React key placement on `.map()` call sites",
    "Resolved all code review issues (tenant-scoped persona query, provider enum validation, model guard, applyEnrichment scope check)"
  ],
  "pains": [
    "Initial attempt at shared enrichment settings state across all notes resulted in unintended global changes.",
    "Used `z.string()` for provider input in tRPC, which was too permissive and insecure.",
    "Attempted Prisma `update` with compound `where` clauses (id + tenantId + projectId), which is not supported for non-unique fields."
  ],
  "successes": [
    "Successfully implemented per-note state management for enrichment settings using `Map`.",
    "Switched to `z.enum(LLM_PROVIDERS)` for strict API validation, improving type safety and security.",
    "Implemented explicit `findFirst` ownership check before Prisma `update` operations, enhancing security and data integrity.",
    "Ensured persona queries are tenant-scoped or built-in, preventing cross-tenant access.",
    "Added robust guards for missing LLM model configurations, providing clear error messages.",
    "Successfully implemented database schema changes and frontend components for action point grouping."
  ],
  "techStack": [
    "TypeScript",
    "React",
    "Next.js",
    "tRPC",
    "Prisma",
    "PostgreSQL",
    "Zod",
    "LLM APIs"
  ]
}