From Markdown to GitHub PRs: Shipping Automated Compliance Reports (and a Little Refactoring Magic)

It was one of those deeply satisfying development sessions where a significant feature goes from concept to deployed code. The mission: build out our automated compliance report export functionality, including the ability to generate a structured report, download it, and even create a GitHub Pull Request for review. And, as often happens, a little refactoring "spring cleaning" snuck in for good measure.

By the end of the day, four solid commits were pushed to main, all tests green, and a new, powerful capability was ready for our users. Let's dive into the details.

The Compliance Report: From Data to Document

The core of this session revolved around giving our users an easy way to export comprehensive compliance reports generated from their workflow runs. This isn't just a simple data dump; these reports need to be structured, readable, and ready for review.

Crafting the Report Formatter

The first piece of the puzzle was creating src/server/services/compliance-report-formatter.ts. This service is the brain behind assembling a rich Markdown compliance report. It takes the raw outputs from our workflow steps and transforms them into a coherent document, including:

• Executive metrics
• Six detailed sections
• Quality gate summaries
• And crucial hallucination/consistency analysis.

The goal here was not just to generate text, but to create a structured and actionable report that stakeholders could easily digest.

The Export Gateway: A tRPC Mutation with a Twist

Next, we exposed this functionality via a tRPC mutation: exportComplianceReport in src/server/trpc/routers/workflows.ts. This mutation is designed to be highly versatile:

1. Download-only mode: Users can simply download the generated Markdown report.
2. Idempotent GitHub PR creation mode: For more integrated workflows, users can opt to have the system automatically create a GitHub Pull Request with the report. This mode is idempotent, meaning repeated calls won't create duplicate PRs, ensuring a smooth user experience even with retries.

Crucially, we baked in robust error wrapping and runtime validation for docRefs (document references), which brings me to a valuable lesson learned later.

A Polished User Experience

No powerful backend feature is complete without a user-friendly frontend. I extracted a self-contained component, src/components/workflow/compliance-export-panel.tsx, to house all the UI logic. This ComplianceExportPanel features:

• An expandable card for a clean UI.
• A dedicated checkbox for enabling GitHub PR creation.
• Clear loading, success, and error states to guide the user.

This panel was then integrated into our dashboard at src/app/(dashboard)/dashboard/workflows/[id]/page.tsx, ensuring it renders only for completed compliance workflows where it's relevant.

Confidence Through Testing

Before even thinking about pushing, a dozen new unit tests were added to tests/unit/compliance-report-formatter.test.ts. All passed, giving us high confidence in the report's generation logic. This is non-negotiable for critical features like compliance reporting.

Under the Hood: Refactoring for a Healthier Codebase

While the main goal was feature delivery, a bit of code hygiene always pays off. I spotted a recurring pattern: multiple implementations of duration formatting.

The formatDuration Consolidation

It's a familiar scenario: you find the same utility function reimplemented in various places, slightly differently. formatDuration was one such culprit. I consolidated six duplicate implementations into a single, shared utility in src/lib/workflow-metrics.ts.

This seemingly small change has a big impact:

• Reduced bundle size: Less redundant code.
• Easier maintenance: Fix a bug once, and it's fixed everywhere.
• Improved consistency: All duration displays will now behave identically.

The removed duplicates were found in: compliance-report-formatter.ts, workflow-bundle.ts, report-generator.ts, model-usage-table.tsx, workflow-performance.tsx, and page.tsx. A true victory for code cleanliness!

Essential Hygiene: Audit Logs & Documentation

Finally, to round out the session, I added:

• Audit logging: For every GitHub PR created by the system, an auditLog() entry (action: compliance_report.pr_created) is now recorded. This is vital for traceability and security.
• Documentation: Generated docs/CONTRIB.md (covering env setup, scripts, dev workflow, testing) and docs/RUNBOOK.md (deployment, monitoring, common issues, rollback, secrets). Good docs are a gift to future-you and your team.

Lessons from the Trenches: Navigating Common Pitfalls

Even with a clear plan, development always throws a few curveballs. These "pain points" are often the most valuable learning opportunities.

1. Validating Prisma JsonValue Arrays: The as Trap

When working with Prisma's JsonValue fields, especially arrays of complex objects, direct as casts can lead to TypeScript errors at compile time or subtle runtime issues.

The Problem: I initially tried to directly cast elements of a JsonValue array (e.g., docRefs) to an expected type or access properties like element.someProperty. This immediately hit TS2339: Property 'someProperty' does not exist on type 'JsonObject | JsonArray'. Prisma's JsonValue is a union type, and TypeScript correctly prevents property access without narrowing.

The Workaround & Lesson Learned: The standard pattern for validating Prisma JsonValue fields at runtime is to first cast the element to Record<string, unknown> (or object), and then use a type guard to check for the presence and type of expected properties.

// Example of a docRef from Prisma JsonValue
interface DocRef {
  id: string;
  url: string;
  // ... other properties
}

function isDocRef(obj: unknown): obj is DocRef {
  return (
    typeof obj === 'object' &&
    obj !== null &&
    'id' in obj && typeof (obj as DocRef).id === 'string' &&
    'url' in obj && typeof (obj as DocRef).url === 'string'
    // ... validate other properties
  );
}

// In your service/mutation logic:
const rawDocRefs = workflow.docRefs as JsonArray; // assuming docRefs is stored as JsonArray
const validatedDocRefs: DocRef[] = [];

for (const rawRef of rawDocRefs) {
  if (isDocRef(rawRef)) {
    validatedDocRefs.push(rawRef);
  } else {
    // Handle invalid docRef, e.g., log error, throw exception
    console.warn('Invalid docRef found:', rawRef);
  }
}

This ensures type safety and robustness against malformed JSON data in your database.

2. Locale-Agnostic Number Testing: The Comma vs. Dot Dilemma

When testing locale-formatted numbers, be wary of hardcoding expected string formats.

The Problem: I had a test assertion like expect(reportContent).toContain("8,000") for a number formatted with Intl.NumberFormat. This worked fine on my machine (which uses a locale with a comma as the thousands separator). However, in a CI/CD environment or on a machine with a different locale (e.g., German), the output would be 8.000, causing the test to fail.

The Workaround & Lesson Learned: For locale-agnostic assertions on formatted numbers, use regular expressions that account for common variations.

// Failed example:
// expect(reportContent).toContain("8,000"); // Fails in 'de-DE' locale

// Robust solution:
expect(reportContent).toMatch(/8[,.]000/); // Matches both "8,000" and "8.000"

This simple regex ensures our tests pass regardless of the environment's locale settings, making our test suite more resilient.

Wrapping Up

This session was a great example of a full-stack feature delivery, touching on everything from backend services and API design to frontend components, robust testing, and crucial refactoring. All 139 tests are passing, typecheck and lint are clean, and the main branch is fully pushed to origin/main.

Next steps involve crucial E2E testing to verify the download and GitHub PR creation flows, and ensuring our broader documentation reflects these new capabilities. It's always satisfying to ship a feature that not only works but is also built on a solid, maintainable foundation.

{"thingsDone":["Created compliance report formatter service","Added exportComplianceReport tRPC mutation (download + GitHub PR)","Developed ComplianceExportPanel component","Integrated ComplianceExportPanel into dashboard","Added unit tests for report formatter","Consolidated formatDuration utility","Added audit logging for PR creation","Generated CONTRIB.md and RUNBOOK.md documentation"], "pains":["TS2339 on Prisma JsonValue array elements","Locale-specific number formatting failing tests"], "successes":["Robust runtime validation for JsonValue","Locale-agnostic test assertions using regex","Significant code duplication reduction","Comprehensive feature delivery with testing and documentation"], "techStack":["TypeScript","Next.js","tRPC","Prisma","React","GitHub API","Jest","Markdown"]}