DATA Remediation: Turning User Incidents Into Agent-Trackable Jobs

A production incident is not useful just because someone reported it. It becomes useful when the product captures enough context to reproduce it, scopes the repair work, gives an agent a job it can safely claim, records what happened, verifies the fix, publishes the result, and tells the submitter that the fix is live. DATA Remediation is VibeChopper's system for making that chain explicit. Send feedback with context

The user-facing version is simple. A creator sends contextual feedback from the editor. They may be describing a broken render, a confusing upload state, a timeline operation that did not behave as expected, or a rough edge they hit while editing by vibe, voice, and precision timeline context. The backend turns that signal into a durable remediation job instead of letting it live as a loose message in a support inbox.

The engineering version is more specific. DATA Remediation jobs have a source type, source ID, incident context snapshot, submitter identity or visitor identity, optional project ID, status, priority, dedupe fingerprint, classification, sanitized prompt, redacted context, allowed file scopes, branch and commit metadata, runner state, check results, publish results, notification results, lock fields, retry fields, and error fields. That sounds like a lot because the system is doing more than storing feedback. It is creating a work item that can survive async agents, admin review, production checks, deploys, notifications, retries, and status-page refreshes.

The primary implementation references are server/dataRemediationRoutes.ts, server/dataRemediationRunner.ts, server/dataRemediationQueue.ts, and the data_remediation_jobs and data_remediation_events tables in shared/schema.ts. The route contract tests in tests/dataRemediationRoutesContract.test.ts also matter because they lock down the public tracker shape: phases, waves, agents, source summaries, and bounded public events.

DATA Remediation starts by accepting that incidents arrive through different doors. The route layer recognizes auto_bug_report, feedback_item, feedback_comment, and voice_feedback_session as source types. Each source can carry different texture. A bug report may have an error message and route. A feedback item may have a product complaint. A comment may point to a discussion thread. A voice feedback session may include a transcript that captures the user's frustration in their own words.

Those differences are valuable, but the repair pipeline needs a common shape. The job queue gives every source the same core fields: source type, source ID, incident context ID, submitter, project, status, priority, and dedupe fingerprint. The source stays traceable, but downstream workers do not need four unrelated job systems. They claim remediation jobs.

Incident context is the bridge. summarizeSource builds a public source summary from source fields and the incident context snapshot. It looks for a title, route, URL, severity, created time, and a bounded summary from fields such as description, message, comment, transcript, error message, feedback context, voice context, or server error context. Public readers get enough context to understand the status without receiving raw internal payloads.

That normalization is what lets VibeChopper treat feedback as product infrastructure. A creator should not need to know whether their report became a database row, an agent prompt, an admin review item, or a status page. They should know that the product captured the context and is tracking the fix. The nearby feedback CTA opens that user-facing path.

The data_remediation_jobs table is intentionally dense. It is the contract between incident capture, worker execution, admin control, publishing, and notification. A thin table with only id, description, and status would be easier to sketch, but it would force every surrounding system to invent side channels. VibeChopper keeps the operational facts on the job. Explore your media graph

Status is the spine. The supported states are queued, classifying, needs_human_review, ready_for_agent, running_agent, awaiting_checks, checks_failed, awaiting_publish, publishing, published, notified, failed, and canceled. That vocabulary is detailed enough to separate planning from building, checks from publishing, and live fixes from notifications. It is also narrow enough for UI, agents, and admins to reason about without parsing arbitrary strings.

Scope is the safety rail. allowedFileScopes gives remediation work a bounded area of the repository. sanitizedPrompt and redactedContext are the bridge between the messy incident and the promptable repair task. branchName, commitSha, pullRequestUrl, and agentRunId connect the job to actual engineering output. runnerStatus, checkResults, publishResult, and notificationResult preserve machine-readable progress across stages.

This mirrors other VibeChopper infrastructure. Upload monitoring, media processing summaries, AI edit runs, and render verification all prefer durable structured records over temporary local state. DATA Remediation applies that same pattern to production repair. If the product is going to let agents help with fixes, the work needs a canonical record that the rest of the platform can inspect.

Internal status names are useful for routing work. They are not always the best language for a submitter watching progress. buildPublicRemediationTracker maps detailed job statuses and worker events into a status tracker with human-scale phases: Received, Planning, Building, Checking, Publishing, and Live. Failed and canceled jobs get explicit terminal states. Send feedback with context

The tracker does more than show a progress bar. It counts events per phase, keeps the latest phase message, records update time, and collects worker waves and agents when events include those fields. Worker callbacks can provide phaseTitle, phaseStatus, waveTitle, agentName, agentRole, rubric, plan, verification, and publicTracker either directly or inside payload. The route normalizes those fields so clients do not have to guess where a worker placed them.

Public data is bounded. Strings are clamped. Arrays are sliced. Nested objects stop after a few levels. The status API returns recent events, not the full private incident record. That restraint matters. A status page should be transparent enough to reduce uncertainty without turning internal logs, private project context, or sensitive repair details into a public dump.

The public endpoint is GET /api/data-remediation/status/:id, with a client page at /data-remediation/status/:id. There is also a source-status endpoint for the original submitter and an authenticated job detail endpoint that expands data for admins. The result is one pipeline with different visibility levels: users see understandable progress, admins see operational context, and workers see claimable jobs.

An agent-trackable job needs more than a queue row. It needs a callback protocol. DATA Remediation gives workers four important routes: claim the next job, append progress events, mark work complete, and record publish results. These are POST /api/data-remediation/worker/claim, POST /api/data-remediation/worker/:id/events, POST /api/data-remediation/worker/:id/complete, and POST /api/data-remediation/worker/:id/publish-result.

The claim route accepts a worker ID and capabilities. The queue claims eligible work, locks it, increments attempts, and returns the job with incident context when available. Event callbacks append an audit trail and can patch job status. Complete callbacks move the job toward awaiting_publish, checks_failed, or failed, while clearing lock fields. Publish-result callbacks record deployed commit SHA, production URL, test results, logs, and errors, then trigger the user notification path when a fix is published.

Callbacks are signed. server/dataRemediationRunner.ts verifies HMAC signatures using DATA_REMEDIATION_WORKER_SHARED_SECRET or REMEDIATION_WORKER_SHARED_SECRET, requires timestamp headers, enforces a five-minute skew window, signs the raw body as timestamp.body, and compares the signature with crypto.timingSafeEqual. That design protects worker endpoints from arbitrary client writes while still allowing headless repair automation to report progress asynchronously.

Dispatch is deliberately controlled. dispatchDataRemediationJob checks ENABLE_DATA_REMEDIATION_AGENT_DISPATCH, a configured SSH command, and a shared secret before spawning a remote remediation runner. If dispatch is disabled or not configured, the function returns a structured skipped or disabled result instead of pretending automation happened. The product can queue and track work even when automatic dispatch is intentionally off.

Agent automation does not remove the need for human gates. DATA Remediation includes authenticated admin routes for listing jobs, filtering by status, source type, and project, approving jobs for agent work, and canceling jobs. A job can sit in needs_human_review until an admin moves it to ready_for_agent. The system is built for repair agents, but it does not assume every incident should immediately modify code.

That distinction is important for creative software. Some user reports are clear defects. Some are feature requests. Some are local browser or media conditions. Some include sensitive project context. Some should become docs, product copy, or manual support follow-up. A remediation pipeline that treats every report as an automatic patch generator will create more risk than value.

The useful approach is narrower: capture rich context, classify it, dedupe it, redact it, scope it, and only then hand work to an agent or a human. The job record supports that approach with classification, dedupeFingerprint, redactedContext, allowedFileScopes, and status transitions that separate queued work from agent-ready work.

This also improves duplicate handling. The schema has a unique source index on source type and source ID, plus a dedupe fingerprint index. That gives the pipeline places to avoid creating repeated jobs for the same source and to find related work. Duplicate incident handling is not glamorous, but it is one of the differences between a toy automation loop and a production repair queue.

Publishing a fix is not the final user experience. The submitter needs to know that the repair passed production checks and is live. When a worker records a successful publish result, DATA Remediation loads the source, sends a fixed notification, merges notification metadata onto the job, and schedules an email fallback. If publishing fails, the system records the failure and sends a failed notification path instead. Create your editing login

There is also an opened signal. POST /api/data-remediation/jobs/:id/opened lets the submitter mark that they opened the notification or status page. The job's notificationResult can record openedAt and openedVia, and the event trail records submitter_opened. That gives the product a way to understand whether the feedback loop actually reached the person who reported the issue.

This is where brand and infrastructure meet. VibeChopper's remediation notifications are direct: your fix passed production checks and is live. The language is simple because the system underneath is specific. The job has publish metadata, production URL, notification metadata, and a public status trail. The message is not empty optimism. It is the user-facing summary of a tracked production repair.

Secure account access matters around this loop, especially when a status page or repaired project context belongs to a creator. The passkeys CTA points to the account path because remediation, notifications, sharing, and cross-device editing all benefit from reliable identity. Anonymous visitors can still be tracked by visitor ID where appropriate, but authenticated users get stronger continuity.

If you are building a feedback-to-fix loop with agents, copy the shape before copying the exact code. Create a durable job. Preserve the source. Store a sanitized agent prompt separately from raw context. Redact what workers do not need. Give jobs a small status vocabulary that maps cleanly to both internal operations and public progress. Keep an append-only event trail. Require signed worker callbacks. Record checks, publishing, and notification results on the job. Open the edit-run receipts

Do not make the agent the source of truth. The agent can build, test, summarize, and report progress, but the server should own job state, authorization, source visibility, callback verification, status transitions, and public payload bounds. That separation keeps automation useful without letting it overwrite the product's operational contract.

Also avoid treating the status page as a vanity feature. Public progress tracking changes how users experience repair. Instead of submitting feedback into silence, they can see that the report was received, planning happened, an agent is building, checks are running, publishing is underway, or the fix is live. That does not make every bug painless, but it makes the product more accountable.

VibeChopper applies the same philosophy to AI edit runs and media graphs. AI-assisted work should leave a trail. A natural-language edit run records plans and tool events. A generated asset records provenance. A render records verification. A remediation job records the path from incident to fix. The surrounding systems differ, but the pattern is the same: keep the work inspectable.

DATA Remediation turns a messy human moment into a structured engineering loop. A creator reports an issue. The system captures context. A durable job records source, project, submitter, status, classification, redaction, scope, and dedupe. A worker can claim the job, post events, attach plans and verification, complete work, report checks, publish a fix, and notify the submitter. The status page translates that activity into phases a user can understand.

That is the correct amount of structure for AI-assisted production repair. The system does not claim that every feedback item should become an automatic code change. It does not expose raw incident context to the public. It does not trust unsigned callbacks. It does not leave the user wondering whether anyone saw their report. It creates a controlled path from incident to fix.

For VibeChopper, this fits the larger product promise. The editor helps creators move faster by letting them describe intent instead of wrestling every timecode by hand. The platform around the editor has to move with the same discipline. When something breaks, the repair path should be as trackable as an edit run, a media asset, or a render artifact.

That is DATA Remediation: capture the context, make the job, run the agent, verify the work, publish the fix, notify the user, and keep the trail.