Automating Compliance: From Report Generation to GitHub Pull Requests

Compliance reports. The words alone can conjure images of tedious manual data gathering, endless copy-pasting, and the looming threat of human error. But what if we could automate this critical process, not just generating reports, but seamlessly integrating them into our existing collaborative workflows?

That was precisely the challenge we tackled in our latest development sprint: building a feature to export compliance analysis reports and, perhaps even more excitingly, automatically create GitHub Pull Requests (PRs) from them. This isn't just about saving time; it's about embedding compliance directly into our development lifecycle, making it auditable, reviewable, and version-controlled.

The Mission: Bridging Compliance and Collaboration

Our goal was clear: empower users to generate a detailed compliance report based on our workflow analysis, and then either download it for local use or push it directly to a specified GitHub repository as a new PR. This would allow teams to review compliance findings alongside their code, discuss remediation strategies, and track changes just like any other development task.

After a focused session, I'm thrilled to report that the core feature is complete, type-checked, lint-passed, and ready for end-to-end testing.

Building Blocks: How We Brought It To Life

This feature touched several parts of our stack, from data formatting to API integration and UI development.

1. The Report Formatter: compliance-report-formatter.ts

At the heart of the operation is our new compliance-report-formatter.ts service. This is a pure data assembly service, meaning it takes raw workflow step outputs (like executive metrics, per-step sections, quality gate results, hallucination analysis, and consistency analysis) and transforms them into a structured Markdown string.

Keeping this service "pure" was a deliberate choice. It ensures that the logic for how a report is structured and formatted is entirely separate from where it's stored or how it's delivered. This makes it highly testable and reusable.

// Simplified conceptual example
export function formatComplianceReport(data: ComplianceWorkflowOutput): string {
  let markdown = `# Compliance Report for Workflow ${data.workflowId}\n\n`;
  markdown += `## Executive Summary\n`;
  markdown += `- **Overall Status**: ${data.overallStatus}\n`;
  // ... more formatting logic
  return markdown;
}

2. The API Gateway: exportComplianceReport Mutation

Next, we exposed this functionality via a new tRPC mutation: exportComplianceReport in src/server/trpc/routers/workflows.ts. This mutation is the orchestrator, accepting parameters for the workflow ID and a crucial createPr boolean flag.

• If createPr is false, the mutation simply returns the Markdown content for direct download.
• If createPr is true, it leverages our existing github-connector.ts to perform a series of GitHub API calls: creating a new branch, committing the generated Markdown file, and finally opening a new pull request.

This dual-mode approach provides flexibility, allowing users to choose their preferred method of report delivery.

3. The User Interface: Compliance Export Panel

Finally, we integrated the feature into our dashboard UI at src/app/(dashboard)/dashboard/workflows/[id]/page.tsx. We designed an expandable card, mirroring our existing BundleExportPanel, which includes:

• A clear button to initiate the export.
• An optional toggle for "Create GitHub Pull Request," which appears only when a linked GitHub repository is configured.

This ensures a consistent user experience and makes the powerful PR creation feature easily accessible.

Refinements & Lessons Learned from Code Review

No feature is truly "done" until it's been through a robust code review. The feedback process was invaluable, leading to several critical improvements:

• Markdown Table Sanitization: We added escapeTableCell() to properly handle special characters (like pipes and newlines) within Markdown table cells, preventing malformed tables in the generated reports. This is a small but vital detail for robust content generation.
• Idempotent GitHub API Flow: When interacting with external APIs, especially GitHub, idempotency is key. We enhanced our GitHub integration to gracefully handle 422 Unprocessable Entity errors (e.g., if a branch already exists) and used getFileSha for file updates to ensure we're always working with the latest version. This prevents errors and ensures reliable PR creation.
• Secure Error Handling: Internal API details should never leak to the client. We wrapped all GitHub API errors in TRPCError instances, providing generic, user-friendly messages while logging the specifics internally.
• Runtime Validation for Prisma Json Fields: Our docRefs field, stored as a Prisma Json type, required careful handling. We implemented runtime validation on both the server and client to ensure its structure matched our expectations, safeguarding against unexpected data shapes.
• Defensive Programming: Replaced potentially unsafe non-null assertions (!) with explicit safe guards in our consistency and hallucination analysis sections. This makes the code more robust and predictable.
• Improved JSX Readability: Extracted a simple isComplianceWorkflow variable for clearer conditional rendering in our JSX, making complex UI logic easier to understand at a glance.

Deep Dive: Taming Prisma's JsonValue with TypeScript

One of the most recurring "pain points" in this session, and frankly, across many of our projects, revolves around working with Prisma's Json fields in TypeScript.

The Challenge: When you retrieve data from a Prisma Json field, TypeScript sees it as Prisma.JsonValue. This type is essentially string | number | boolean | Prisma.JsonObject | Prisma.JsonArray | null, where JsonObject is {[key: string]: Prisma.JsonValue} and JsonArray is Array<Prisma.JsonValue>.

The problem arises when you have an array of objects stored in a Json field, like our docRefs (e.g., [{ owner: 'org', repo: 'repo', path: 'file.md' }]). If you try to directly access properties on an element from this array, TypeScript throws TS2339: Property 'owner' does not exist on type 'JsonObject | JsonArray'. This is because Prisma.JsonValue itself doesn't guarantee property access, and even Prisma.JsonArray's elements are still Prisma.JsonValue.

The Failed Attempt: My initial instinct was often to use a direct as cast on the array elements:

// This causes TS2339
const docRef = (myJsonArray[0] as { owner: string; repo: string; path: string });
console.log(docRef.owner); // Error!

The Workaround & Lesson Learned: The reliable pattern, which we now consistently apply, is to first cast the element to Record<string, unknown> and then use a type guard to safely check for the presence and type of the desired properties.

type DocRef = { owner: string; repo: string; path: string };

function isDocRef(obj: unknown): obj is DocRef {
  if (typeof obj !== 'object' || obj === null) {
    return false;
  }
  const record = obj as Record<string, unknown>; // Intermediate cast
  return typeof record.owner === 'string' &&
         typeof record.repo === 'string' &&
         typeof record.path === 'string';
}

// In our application code:
const rawDocRefs = workflow.docRefs as Prisma.JsonArray; // Assume it's an array
const validatedDocRefs: DocRef[] = [];

for (const item of rawDocRefs) {
  if (isDocRef(item)) {
    validatedDocRefs.push(item);
  } else {
    console.warn("Invalid docRef found:", item);
    // Handle invalid item, e.g., skip or throw
  }
}

// Now you can safely use validatedDocRefs
validatedDocRefs.forEach(ref => console.log(ref.owner));

This ensures type safety and robustness when dealing with dynamic Json data, preventing runtime errors due to unexpected data shapes. It's a recurring pattern, and having a clear strategy for it saves a lot of headaches.

What's Next?

While the core feature is complete, there are always next steps to refine and enhance:

1. End-to-End Testing: Thoroughly test the entire flow, from running a compliance workflow to verifying Markdown downloads and successful GitHub PR creation.
2. UI Component Extraction: The dashboard page file is growing. We'll extract the compliance export panel into its own dedicated component (src/components/workflow/compliance-export-panel.tsx) to improve maintainability and reduce file size.
3. Audit Logging: Implement audit logging for GitHub PR creation actions, providing a clear trail of who initiated what and when.

This feature marks a significant step forward in automating and integrating our compliance workflows. By leveraging our existing tools and applying careful design principles, we're making compliance less of a burden and more of an integral, collaborative part of our development process.