Building a Knowledge Hub: From CRUD to Intelligence with 4 Data Sources

Last week, I tackled an interesting challenge: transforming a basic 194-line CRUD memory list into something much more powerful—a unified Knowledge Hub that could intelligently surface insights from four different data sources.

The Challenge

Our /dashboard/memory page was functional but uninspiring. It displayed memory entries in a simple list format, but we had so much more data sitting in our database:

• MemoryEntry - User notes and observations
• WorkflowInsight - Process improvements and bottlenecks
• ConsolidationPattern - Data consolidation strategies
• CodePattern - Reusable code snippets and solutions

The goal was ambitious: create a unified interface that could cluster related knowledge, provide smart suggestions, and enable semantic search across all these data sources.

The Architecture

Unified Data Model

First, I needed to normalize these four different data structures into a common interface. I created a KnowledgeItem type that could represent any piece of knowledge regardless of its source:

// src/types/knowledge.ts
export interface KnowledgeItem {
  id: string;
  title: string;
  content: string;
  source: 'memory' | 'insight' | 'consolidation' | 'code';
  cluster: 'issues' | 'solutions' | 'patterns' | 'tools' | 'notes';
  severity?: 'low' | 'medium' | 'high' | 'critical';
  tags: string[];
  metadata: Record<string, any>;
  createdAt: Date;
  updatedAt: Date;
}

Smart Clustering

The real magic happened in the clustering logic. I implemented a getCluster() function that intelligently categorizes knowledge based on keywords and context:

export function getCluster(title: string, content: string, source: string): KnowledgeCluster {
  const text = `${title} ${content}`.toLowerCase();
  
  if (text.includes('error') || text.includes('bug') || text.includes('issue')) {
    return 'issues';
  }
  if (text.includes('solution') || text.includes('fix') || text.includes('resolve')) {
    return 'solutions';
  }
  // ... more clustering logic
}

Service Layer Design

The knowledge-hub.ts service became the orchestrator, with normalizer functions for each data source:

// Normalize different data sources into KnowledgeItems
const normalizeMemoryEntry = (entry: MemoryEntry): KnowledgeItem => ({
  id: entry.id,
  title: entry.title,
  content: entry.content,
  source: 'memory',
  cluster: getCluster(entry.title, entry.content, 'memory'),
  // ... rest of normalization
});

The queryKnowledgeHub() function handles the complex task of querying multiple tables, normalizing results, and applying filters—all while maintaining good performance.

The User Experience

Three-Tab Layout

I redesigned the page with a clean three-tab interface:

1. Knowledge Tab - Clustered view of all knowledge items
2. Suggestions Tab - AI-generated insights and recommendations
3. Entries Tab - Preserved the original CRUD timeline for power users

Intelligent Features

Clustered Views: Knowledge items are automatically grouped into logical clusters (Issues, Solutions, Patterns, Tools, Notes) with collapsible sections.

Smart Search: Users can toggle between keyword search and semantic search (powered by vector embeddings) with a simple sparkles icon.

Related Knowledge: Each knowledge item includes a "Find related" button that uses vector similarity to surface connected insights.

Visual Analytics: Stats dashboard shows Knowledge Assets, Solutions & Patterns, Resolution Rate, and Intelligence Coverage with positive, marketing-friendly framing.

Component Architecture

I built 11 specialized React components, each with a single responsibility:

• knowledge-search.tsx - Debounced search with semantic toggle
• knowledge-filter-bar.tsx - Multi-dimensional filtering (source, cluster, severity, project)
• knowledge-item-card.tsx - Expandable cards with rich metadata
• knowledge-cluster-group.tsx - Collapsible cluster sections
• knowledge-stats.tsx - Analytics dashboard
• And 6 more specialized components...

Each component is designed to be reusable and testable, following React best practices.

Lessons Learned

The Client-Server Boundary Challenge

The biggest technical hurdle came from Next.js's strict client-server separation. I initially tried to import the getCluster() function from the server service into a client component:

// ❌ This breaks the build
import { getCluster } from 'src/server/services/knowledge-hub.ts'

The error was cryptic but the cause was clear: the server service imports PrismaClient, which can't be bundled for the client.

Solution: I moved all shared logic to src/types/knowledge.ts—a pure functions file with no server dependencies. Both client components and server services can safely import from there.

Key Takeaway: Any logic shared between client and server must live in src/types/ or src/lib/, never in src/server/.

Performance Considerations

With four data sources and complex filtering, I had to be thoughtful about performance:

• Implemented proper debouncing on search inputs
• Added skeleton loading states for better perceived performance
• Capped results at 100 items with plans for pagination
• Used React.memo() strategically on expensive components

The Results

The transformation was dramatic. What started as a simple list became an intelligent knowledge management system that:

• Unifies 4 disparate data sources
• Automatically clusters related knowledge
• Provides semantic search capabilities
• Generates contextual suggestions
• Maintains the original CRUD functionality for power users

The new Knowledge Hub doesn't just display data—it creates connections, surfaces insights, and helps users discover knowledge they didn't know they had.

What's Next

The foundation is solid, but there's room for enhancement:

• Mobile optimization - Better responsive design for stacked filters
• Advanced pagination - Handle large datasets more elegantly
• Machine learning - Improve clustering with trained models
• Collaboration features - Enable knowledge sharing between team members

Building this Knowledge Hub reminded me why I love full-stack development: the opportunity to create systems that are not just functional, but genuinely intelligent and helpful. Sometimes the best features are the ones that connect existing data in new ways.

Want to see more architecture deep-dives? Follow me for more posts on building intelligent developer tools and dashboard experiences.