Unlocking Smarter LLM Pipelines: Injecting Project Wisdom for AutoFix and Refactor

The dream of truly intelligent development tools often hinges on one critical factor: context. Large Language Models (LLMs) are incredibly powerful, but their generic nature can sometimes lead to suggestions that, while syntactically correct, miss the mark on project-specific nuances, architectural patterns, or established best practices.

Today, we're thrilled to announce a significant leap forward in our AutoFix and Refactor pipelines. We've officially made them project-aware, injecting a consolidated stream of project-specific knowledge directly into the LLM prompts. This means our pipelines now understand your project's wisdom – its documentation, past discussions, previous runs, and collective memory – leading to far more relevant, accurate, and actionable code improvements.

This feature is now fully implemented, type-checked, and live on main as of 9a632ac. Let's dive into the journey of building this intelligent context injection system.

The "Why": Beyond Generic Suggestions

Imagine an LLM suggesting a code change. If it only sees the snippet in isolation, it might propose a valid fix. But what if your project has a specific library preference, a mandated architectural pattern, or a previous design decision documented in a discussion? A generic LLM won't know.

Our goal was to bridge this gap. By consolidating knowledge from various sources – wisdom, memory, discussions, documentation, and even insights from previous pipeline runs – and injecting it into the LLM's prompt, we empower it to act as a truly informed team member, not just a powerful autocomplete engine.

Building the Brain: A Technical Deep Dive

Bringing project awareness to our pipelines required a multi-faceted approach, touching both our backend services and the user-facing frontend.

1. Laying the Foundation: Project Association

The first step was fundamental: associating every AutoFixRun and RefactorRun with a specific projectId. This involved adding a projectId foreign key to these models in our prisma/schema.prisma and establishing reverse relations on the Project model. This simple database change became the anchor for all subsequent context-aware operations.

typescript

// Excerpt from prisma/schema.prisma
model Project {
  id           String        @id @default(cuid())
  name         String
  // ... other project fields
  autoFixRuns  AutoFixRun[]
  refactorRuns RefactorRun[]
}

model AutoFixRun {
  id        String    @id @default(cuid())
  projectId String?
  project   Project?  @relation(fields: [projectId], references: [id])
  // ... other AutoFixRun fields
}

2. The Context Engine: `pipeline-context.ts`

The heart of this new feature lives in src/server/services/pipeline-context.ts. This service is responsible for orchestrating the retrieval and consolidation of project knowledge. It pulls data from five distinct sources:

Wisdom: Curated project-specific guidelines.
Memory: Past interactions and learned insights.
Discussions: Relevant conversations and decisions.
Documentation: Project-specific docs and READMEs.
Previous Runs: Insights from how past AutoFix or Refactor runs performed.

To manage the token limits inherent in LLM prompts, this service also intelligently caps the total consolidated knowledge at approximately 30,000 characters. This ensures we provide rich context without overwhelming the model or incurring excessive costs.

3. Orchestrating the Flow: Backend Integration

With the context engine in place, we needed to ensure this valuable information flowed correctly through our system:

Frontend-to-Backend Handoff: We extended the start mutations in our tRPC routers (src/server/trpc/routers/auto-fix.ts and refactor.ts) to accept projectId, memoryIds, and contextSources as inputs. This allows the frontend to specify exactly which project and context elements to use for a given pipeline run.
Pipeline Orchestration: Both our pipeline orchestrators (auto-fix/pipeline.ts and refactor/pipeline.ts) were updated to call loadPipelineContext() and pass the resulting projectContext downstream.
LLM Prompt Injection: Crucially, all four of our core LLM components – issue-detector.ts, fix-generator.ts, opportunity-detector.ts, and improvement-generator.ts – were modified to accept and inject the projectContext directly into their prompts. This is where the magic happens, transforming generic LLM calls into project-aware ones.
Real-time Updates: Our Server-Sent Events (SSE) routes (api/v1/events/auto-fix/[id]/route.ts and refactor/[id]/route.ts) were updated to correctly pass configuration fields to the pipelines, ensuring real-time feedback reflects the chosen context.

4. A User-Friendly Interface: Frontend Enhancements

A powerful backend feature is only as good as its user interface. We overhauled our frontend list pages to provide intuitive controls for this new capability:

Project Selector Dropdown: Easily switch between projects to view relevant runs.
Context Source Toggle Chips: Granular control over which context sources (wisdom, memory, docs, etc.) are active for a new run.
Collapsible MemoryPicker: Select specific memory insights to include.
Filtered Repo Dropdown: Repositories are now filtered based on the selected project.

On the detail pages (auto-fix/[id]/page.tsx and refactor/[id]/page.tsx), we added context badges displaying the project name, active source icons, and memory count. This provides immediate visual confirmation of the context used for any given pipeline run.

Navigating the Rapids: Lessons Learned

Development isn't always smooth sailing. Here are a couple of key challenges we encountered and how we overcame them:

1. Prisma and Custom Database Types: The `embedding vector` Dilemma

Challenge: After adding projectId to our prisma/schema.prisma, running npm run db:push (which applies schema changes) triggered a warning about dropping the embedding vector(1536) column on our workflow_insights table. This column stores vector embeddings for RAG (Retrieval Augmented Generation) and is managed via raw SQL, as Prisma doesn't natively support the vector type.

Lesson Learned: When mixing Prisma with custom, unsupported database types, schema migrations can be tricky. Prisma's db:push (or migrate dev) will try to reconcile the database schema with its internal model, potentially dropping columns it doesn't recognize or manage.

Workaround: We had to use the --accept-data-loss flag with db:push and then immediately restore the embedding column and its associated HNSW index using raw SQL commands via ALTER TABLE workflow_insights ADD COLUMN IF NOT EXISTS embedding vector(1536) and npx prisma db execute --stdin. This is a recurring issue, and we've learned to always anticipate and restore this column after schema pushes.

2. Frontend Data Structure Awareness: `data` vs `data.items`

Challenge: In our frontend, we initially tried projects.data?.map() to render a list of projects from a tRPC query. TypeScript correctly flagged this as an error.

Lesson Learned: Always double-check the exact structure of API responses! Our trpc.projects.list query returns an object with an items array and a total count, not just an array directly.

Workaround: The fix was simple: projects.data?.items.map(). A good reminder that even experienced developers can sometimes overlook fundamental API response structures, and TypeScript is a fantastic guardrail.

Looking Ahead

With the core project-awareness feature deployed, our immediate next steps involve thorough quality assurance to ensure backward compatibility and robust functionality with and without project context. We'll be verifying everything from repo filtering to context badge display. Beyond that, we're excited to continue enhancing our RLS (Row Level Security) policies and tidying up our development environment.

This release marks a significant step towards truly intelligent, context-aware development tools. We believe that by empowering LLMs with the unique wisdom of your projects, we can help developers build better software, faster. Try it out and let us know what you think!