Unlocking Workflow Wisdom: Building Our Project Memory System

Every developer knows the feeling: you've built something great, learned valuable lessons, and then... you move on. Those insights often get lost, only to be rediscovered (or reinvented) later. What if your development workflow could remember its own past, allowing you to easily reuse hard-won knowledge? That's the core idea behind the "project-workflow memory system" we've been building, and I'm thrilled to share that we've just completed a significant phase!

This past session was all about bringing this memory system to life, from the front-end user experience to the backend plumbing. We focused on making it effortless to capture, manage, and inject past insights directly into new workflows.

The Building Blocks of Memory: What We Shipped

Our goal was clear: complete the memory system, making it robust and user-friendly. Here's a breakdown of the key components we brought to fruition:

1. The MemoryPicker: Injecting Past Wisdom

At the heart of our system is the MemoryPicker component. Imagine you're starting a new workflow and want to leverage insights from similar past projects. The MemoryPicker makes this seamless:

• Intelligent Search: A powerful search bar helps you quickly find relevant insights.
• Category Filters: Chips allow you to narrow down insights by category.
• Severity-Colored Badges: Visual cues (like severity-colored badges) help prioritize or understand the nature of each insight at a glance.
• Expandable Details: You can expand an insight to see its full details and a suggestion for how it might be used.
• {{memory}} Injection Preview: Crucially, it provides a real-time preview of how the selected insight will be injected into your workflow prompt using a {{memory}} template placeholder.

This component queries our memory.listInsights tRPC endpoint, ensuring you always have access to the latest collective wisdom. We then integrated this into our workflow creation page (new/page.tsx), wiring up the selected memoryIds to the create mutation.

2. Streamlining Our Backend: The Power of Consolidation

Maintaining a clean and efficient API is crucial. We identified a duplicate saveInsights mutation – one in src/server/trpc/routers/workflows.ts and another, more robust version, in memory.saveInsights. We decided to consolidate, removing the duplicate and ensuring all insight-saving operations now go through the richer, more validated trpc.memory.saveInsights endpoint. This not only cleans up our codebase but also centralizes validation logic, making our system more robust.

3. Enhancing Workflow Clarity with Collapsible Sections

Complex workflows can quickly become visually overwhelming. To combat this, we introduced a new CollapsibleSection component. Now, sections like "Consolidations," "Personas," "Docs," and "Memory Insights" are all collapsible by default. A small badge in the header conveniently shows the number of selected items within that section, giving users a quick overview without needing to expand everything. This is a small but mighty UX improvement for managing information density.

4. The Proof is in the Pudding: End-to-End Validation

No feature set is truly complete without rigorous testing. We ran a full end-to-end test:

• Created a new workflow.
• Progressed through "Analyze," "Design Features," and "Review" steps.
• Approved the review, which extracted 10 key points.
• The SaveInsightsDialog appeared (more on why this was a win below!).
• Successfully saved all 10 insights to our workflow_insights table.
• Finally, verified that these new insights were immediately visible in the MemoryPicker on a new workflow creation page.

Seeing the entire cycle work flawlessly was incredibly satisfying!

Navigating the Treacherous Waters: Lessons Learned

Development isn't always a smooth sail. We hit a couple of snags that provided valuable lessons.

Lesson 1: The Elusive "Approve & Continue" Bug

The Problem: After completing a review step, clicking "Approve & Continue" was supposed to bring up the SaveInsightsDialog. Instead, it skipped directly to the next step, leaving our valuable insights unsaved.

The Root Cause: Our extractKeyPoints() function was working as intended, but the extracted key points simply didn't have an action field – it was undefined. Our filter, kp.action === "keep" || kp.action === "edit", therefore evaluated to false for all key points, leading the dialog to believe there was nothing to save.

The Fix: A subtle but critical change to the filter condition in both save-insights-dialog.tsx and workflows/[id]/page.tsx. We updated it to !kp.action || kp.action === "keep" || kp.action === "edit". This now correctly interprets undefined (or null) actions as saveable, alongside "keep" or "edit" actions. It's a classic example of how unexpected undefined values can derail an otherwise perfectly logical conditional.

Lesson 2: The Case of the Duplicate Saves

The Problem: During testing, we noticed clicking the "Save insights" button multiple times resulted in duplicate records (sometimes 30 records instead of 10). This pointed to a potential race condition or an oversight in our UI/UX.

The Solution (and Next Steps): While we temporarily cleaned up the duplicates with a quick SQL query, this highlights a critical area for improvement. For immediate prevention, we'll be adding a guard to disable the save button after the first click. This is a simple yet effective UX pattern to prevent accidental duplicate submissions. For a more robust solution, we'll also investigate mutation deduping strategies on the backend to handle concurrent requests gracefully.

What's Next on Our Horizon?

With the core memory system complete and stable, our sights are set on the next phase of enhancements:

1. Duplicate Save Guard: Implement the UI-level disabled guard for the save button.
2. Vector Search Integration: This is a big one! We'll be updating our Docker Postgres instance to use pgvector/pgvector:pg16, installing the pgvector extension, and adding an embedding column to our insights. This will unlock powerful vector similarity search, making our memory system even smarter and more relevant.
3. {{memory}} Template Resolution Test: We need to thoroughly test that selecting memory insights and injecting them via {{memory}} placeholders correctly resolves within the workflow execution.
4. Project-Scoped Insight Filtering: To further enhance relevance, we'll consider adding a projectId filter to the MemoryPicker, ensuring users primarily see insights pertinent to their current project.
5. Auto-Pull Memory Files from Git: A highly requested feature is to automatically pull .memory/ files from Git repositories, allowing our system to passively learn from code changes and documentation.

Conclusion

This session marked a significant leap forward for our project. We've built a robust, user-friendly system that empowers developers to capture, manage, and reuse valuable insights, transforming our workflows into a continuous learning loop. Tackling bugs and streamlining our backend were crucial parts of this journey, reinforcing the importance of thorough testing and thoughtful architecture. We're incredibly excited about the potential of this memory system and the intelligent capabilities that pgvector will soon unlock!

---