RESEARCH

Findings we're willing to defend

Mifa publishes applied research from production systems: agent reliability, evaluation methodology, document intelligence, and the governance of autonomy. Numbers come from real deployments, caveats included — because research you can't reproduce is just marketing with citations.

PUBLICATIONS

Selected technical reports

Full PDFs, datasets where releasable, and reference implementations are available on request — or bundled with enterprise partnerships.

Mifa Technical Report · 2026 Agentic systems

Supervision Trees for Multi-Agent Workflows: Failure Containment in Production Agent Fleets

Vogel, L., Reyes, D., et al.

Abstract. We adapt hierarchical supervision — long used in fault-tolerant telecom systems — to fleets of LLM-based agents. Across 14 months of production traces from three enterprise deployments, supervised topologies reduced unrecoverable workflow failures by 83% relative to flat orchestration, at a median latency overhead of 4.1%. We characterize which failure classes supervision cannot contain and propose escalation contracts for human-gated recovery.

Mifa Technical Report · 2026 Evaluation

Trace-Distilled Evaluation: Mining Production Logs for Regression Suites That Predict Incidents

Vogel, L., Ishikawa, M., Adeoye, T.

Abstract. Hand-curated evaluation sets systematically under-sample the failure modes that cause production incidents. We present a pipeline that distills anonymized production traces into compact regression suites, weighting by downstream error cost. Suites built this way predicted incident-causing regressions 4.2x more reliably than expert-curated sets of equal size across two contact-center deployments.

Mifa Technical Report · 2025 Document intelligence

Layout Robustness in Structured Document Extraction: Measuring and Mitigating Format Drift

Ishikawa, M., Vogel, L., Okoro, C.

Abstract. Document extraction systems degrade silently when upstream forms are redesigned — a failure mode we term format drift. We introduce a drift benchmark built from 12,000 synthetically re-laid-out enterprise documents and show that layout-randomized fine-tuning recovers 91% of baseline accuracy under severe drift, versus 62% for models trained on canonical layouts alone.

Mifa Technical Report · 2025 Systems

Learned Cost-Quality Routing for Heterogeneous LLM Backends

Reyes, D., Adeoye, T., Vogel, L.

Abstract. We formulate model selection as a constrained routing problem: minimize inference cost subject to a per-task quality floor estimated from evaluation history. A lightweight learned router held 96.3% of frontier-model task quality at 38% of the cost across three production workloads, and degraded gracefully under distribution shift when paired with an abstention fallback.

Mifa Technical Report · 2025 Governance

What Would It Take to Audit an Agent? Requirements from Regulated-Industry Deployments

Vogel, L., Whitcombe, R., Mehta, S.

Abstract. Drawing on deployment reviews with compliance teams at four regulated enterprises, we catalogue the evidentiary requirements — action provenance, authorization context, replayability, and counterfactual traces — that autonomous systems must satisfy for internal audit sign-off. We propose a hash-chained action-log schema meeting these requirements and measure its overhead in a claims-processing pilot.

Mifa Technical Report · 2024 Human-AI systems

The Economics of Human-in-the-Loop: When Review Queues Help, and When They Hide Failure

Okoro, C., Reyes, D.

Abstract. Human review is treated as a universal safety valve for AI systems, but poorly designed queues create rubber-stamp dynamics that are worse than automation alone. Using 400,000 review decisions from two document-processing deployments, we identify queue designs that preserve reviewer vigilance — confidence-stratified sampling, seeded known-error items — and quantify the cost-accuracy frontier for each.

OPEN SOURCE

Tools we maintain in the open

The evaluation and observability tooling our engagements depend on — open-sourced because trust infrastructure works better when everyone can read it.

mifa/evalgate

A CI-native harness for regression-gated LLM evaluation: define quality bars as code, block deploys that regress them. Our most-adopted tool — used well beyond our own client base.

Python CI/CD · evaluation

mifa/tracelens

Structured capture and replay for agent execution traces — every tool call, authorization, and model response, hash-chained for tamper evidence and replayable for audit.

TypeScript Observability · audit

mifa/driftwatch

Behavioral drift monitoring for production language systems. Compares live output distributions against eval baselines and pages you before your users notice.

Python Monitoring · MLOps

mifa/formforge

Synthetic document generator for extraction testing: takes a schema and emits thousands of realistically messy layout variants — the tooling behind our format-drift benchmark.

Python Synthetic data · testing

PRINCIPLES

How we keep the research honest

Four rules every Labs project signs up to before the first experiment runs.

01 Publish the failures

Negative results are findings. Internal experiment reports record what didn't work and why, and our public write-ups keep the caveats the marketing team would rather cut.

02 Production data or honest synthetics

Benchmarks built on toy datasets produce toy conclusions. We evaluate on anonymized production traces where consent allows, and on synthetics engineered to preserve the ugliness of the real thing everywhere else.

03 Client confidentiality is absolute

Everything we publish is aggregated, anonymized, and released only with client review. No paper is worth a client's trust, and none has ever cost us one.

04 Research serves deployment

We are an applied lab. Every research thread must name the engagement problem it serves, and every published technique ships with a reference implementation someone can actually run.

Want the full papers?

Request any technical report, dataset card, or reference implementation — or talk to the researchers directly about applying the findings to your systems.

Request research access Visit AI Labs