# MICA — Methodology & References

Generated from the canonical wiki records in `src/data/methodology/wiki.ts`. Do not edit this file by hand: run `pnpm export:data`.

This document states MICA's current final methodology and its evidence base.

## Current status

- Canonical public domain: micabench.com
- Markets: 6 (KR, JP, SG, TW, AE, TH)
- Task families: 10
- Canonical task definitions: 100 (100 provisional public-set candidates, 0 validated)
- Canonical task text languages: en, ko
- Registered systems: 0
- Run cells: 0
- Publication eligible: false
- Minimum eligible runs: not set
- Minimum coverage: not set
- Per-task final score: 100 × accuracy × speed × cost, each factor normalized to the interval from 0 to 1, bounded by 0 and 100
- References in this document: 56

## Reading vocabulary

- **Current rule.** A rule MICA applies today. Binding on the method as it stands.
- **MICA policy.** A normative choice MICA made for itself. No external work endorses it, and it is open to challenge on its own terms.
- **External evidence.** A finding or practice taken from published work, cited in the reference chapters. Borrowing is not endorsement by the cited authors.
- **Open question.** Not settled. MICA states the question rather than implying an answer it does not have.

## Why MICA exists

MICA is the world's first public benchmark initiative designed for outcome-verified end-to-end consumer-agent execution across 6 country-localized real-life markets and multiple everyday consumer domains. Published agent benchmarks measure capability in environments the benchmark controls, while the thing a household actually buys is a whole system working inside local services nobody controls. MICA exists in that gap, and it is an initiative designed to evaluate rather than a benchmark reporting results.

- **[External evidence] The world-first claim, and its scope.** MICA is the world's first public benchmark initiative designed for outcome-verified end-to-end consumer-agent execution across 6 country-localized real-life markets and multiple everyday consumer domains. The claim is the conjunction of four conditions at once: 6 country-localized real-life markets, multiple everyday consumer domains, end-to-end execution rather than navigation or answering, and an authoritative outcome or final-state evidence requirement. It is a scoped category claim. It does not say prior work lacked live websites, shopping, travel, apps, transactions or state checks individually, because prior work has each of those. Review cutoff is 8 August 2026, and the claim is made to our knowledge from a finite literature review. MICA is currently an initiative designed to evaluate, with 0 systems and 0 run cells, so it is not a benchmark reporting measured superiority over anything. (see: webarena, webvoyager, webshop, androidworld, appworld, tau-bench)
- **[External evidence] The measurable gap.** Execution-based benchmarks established that a task can be graded on final state rather than on a transcript, and they are not confined to controlled environments: WebVoyager operates on live websites, WebArena, WebShop, AndroidWorld and AppWorld use controlled or simulated ones, and tau-bench judges interaction against simulated domain policy and backend state. What none of them assembles is the conjunction: 6 country-localized real-life markets, multiple everyday consumer domains, execution carried through to the end, and an authoritative completion record, in the language and register the user actually uses. (see: webarena, webvoyager, webshop, androidworld, appworld, tau-bench, osworld)
- **[Current rule] The system is the subject.** A base model score does not predict whether an errand completes. Scaffolding, routing, memory, permissions and recovery decide that, and they are the parts a buyer chooses between. MICA therefore measures the assembled system and never a model in isolation.
- **[MICA policy] Locality is the hard part.** A benchmark that translates English tasks measures translation. MICA's markets differ in which channel a journey lives on, who has to approve it, when it is actually complete and what counts as proof, and those differences are the point rather than a complication. (see: blend, m3exam, mega)
- **[Current rule] What MICA refuses to be.** Not a marketing surface for the systems it measures, not a leaderboard that publishes a figure it cannot re-derive, and not an index that treats an absent measurement as a low one.

## Construct and evaluation unit

What exactly is being measured, stated precisely enough that a submitter knows what to freeze and a reader knows what a score is about.

- **[Current rule] The evaluation unit.** A complete versioned consumer-agent system snapshot. It includes the scaffold and orchestration, task-specific model routing, memory, APIs, tools, browser and app operation, permissions, localization, safety behaviour and recovery. It is never a base model.
- **[Current rule] Versioning obligation.** A snapshot is identified by a stated version and date. A system that changes mid-evaluation is a different snapshot, and results do not carry across.
- **[External evidence] Why not a model score.** Heterogeneous interactive benchmarks already show that per-environment behaviour diverges sharply for systems sharing a base model, which is why aggregation across unlike settings is avoided and the assembled system is the subject. (see: agentbench, helm)
- **[Current rule] Declared composition.** A submitted snapshot declares its orchestrator, the models it may route to, the tools it may call and its memory behaviour. An undeclared capability used during a run invalidates the run.

## Current benchmark lifecycle and status

Where the index actually stands. Every figure in this chapter is counted from the live records rather than written into the copy.

- **[Current rule] Scope in force.** 6 markets, 10 task families and 100 canonical task definitions. That is the whole of the current scope.
- **[Current rule] No measured results.** 0 systems are registered and 0 run cells exist. Publication eligibility is false. Nothing on this site describes how any system behaves.
- **[Current rule] Catalogue status.** All 100 public definitions are provisional candidates and 0 are validated. Canonical task text exists in en and ko only, which is translation support and not linguistic validation of every market.
- **[MICA policy] Candidates are design material.** The published definitions exist so the method can be criticized before it produces a number. They must not be reverse-engineered into the final benchmark: future task authoring follows this methodology independently of them.
- **[Current rule] Fail-closed publication.** Minimum sample size and minimum coverage are unset, and the gate refuses rather than assuming a value. Demo and preview data can never become publication eligible, which the schema layer enforces at build time.

## What is implemented now, and what is planned

A design intent stated confidently reads as a shipped capability. This chapter separates the two item by item, and nothing marked planned exists.

- **Implemented — Bilingual public site and published method** This wiki, the market editions, the task catalogue and the machine-readable exports are live and generated from typed source data.
- **Implemented — Validated data and publication contract** Schema validation, holdout separation on export, the fail-closed publication gate and the per-task scoring functions all run in code and fail the build when violated.
- **Implemented — Declared market integration profiles** Every market edition declares representative surfaces, authorization boundaries, completion semantics, recovery conditions and evidence requirements. Declared coverage is a plan, not a measurement.

- **Planned — Execution harness and evaluators** A resettable environment, a run recorder, deterministic final-state evaluators and a cost and latency ledger. None of it exists yet, so no attempt has ever been executed.
- **Planned — Validated localized scenarios** Locally authored and locally reviewed executable scenarios with pre-registered speed and cost references. The current catalogue is candidate definitions only.
- **Planned — Private holdout set** The separation is enforced in the export today, but no holdout item has been authored, so there is nothing being held back yet.
- **Planned — Repeated runs and uncertainty reporting** Interval estimates, repeated-trial reliability and sensitivity analysis over the scoring policy. All of it waits on the harness.
- **Planned — Live shadow and limited verified-live tracks** No vendor integration exists, no live account is held and no run has been performed against a real service.
- **Planned — Publication thresholds** Minimum sample size and minimum coverage are unset by design. Until they are agreed, the gate refuses every result rather than guessing a number.
- **Planned — Independent decision body** No committee, board or review panel has been constituted, and no membership exists. Nothing on this site should be read as the output of one.

## Task authoring and market localization

How a task becomes a MICA task, and what makes it local rather than translated.

- **[Current rule] Three orthogonal design fields.** Every task declares an interaction surface, a termination or completion class, and a declared cognitive complexity. They vary independently, so difficulty cannot be smuggled in through the surface, and the catalogue can be balanced on each field separately.
- **[MICA policy] Locally authored, locally reviewed.** A task is authored and reviewed by someone who uses the market's services. Translation of an English task is not localization, and a task that survives translation unchanged is usually testing nothing local. (see: m3exam, blend, data-statements)
- **[Current rule] What locality has to cover.** Actual service and channel conventions, address and name formats, date, time and currency handling, language register and script, holidays, payment and identity rails, the points where a user must take over, and what makes a final state valid in that market.
- **[Open question] The current translation limit.** Canonical task text exists in en and ko. Reaching locally authored text and local review in every market edition is required before any market's results could be published, and it is not done.
- **[External evidence] Documentation per edition.** Each catalogue edition carries a datasheet-style record of motivation, composition, collection, intended uses, distribution and maintenance, plus a data-statement record of language variety and reviewer context at role level. (see: datasheets, data-statements)
- **[External evidence] Parameterized instances.** One definition should yield many checkable instances rather than a single memorizable script, following parameterized-task practice from business and mobile agent environments. (see: workarena, androidworld)

## Access parity and market integration conditions

If two systems face different starting conditions, the comparison measures the conditions. MICA fixes them, and declares what each market edition will require.

- **[Current rule] What is fixed equally.** The selected representative platforms, the account tier, the granted permissions, the initial state and the confirmation boundary are identical across systems for a given task version. (see: mlperf-inference)
- **[Current rule] Enclosure is a condition, not a bonus.** App enclosure and identity handoff are controlled conditions of the environment. A system is not awarded extra credit for operating inside an enclosed channel, and is not excused for failing to.
- **[External evidence] Execution is platform-controlled, separately from whether an API exists.** Official platform documentation shows that discovery and read access is often credentialed but broadly documented, while state-changing execution is conditioned. Messaging, payment, account operations and mini-app execution can require a registered application or channel, OAuth or platform consent, reviewed scopes, a permitted recipient relationship, merchant or partner onboarding, a user gesture, a controlled runtime and a release review. None of that means a market has no APIs; it means the presence of an API does not establish that a consumer errand can be completed in production by a given system on a given date. MICA therefore records production executability as a separate fact from API existence, per market edition, and treats it as a condition of the environment rather than a property of a system. (see: kakao-app, kakao-message, line-channels, line-messaging, line-mini-app, wechat-release, grab-docs)
- **[Current rule] Market integration profiles.** Each of the 6 market editions declares at least 5 representative situations, spanning at least 3 surfaces, 3 authorization boundaries, 2 completion semantics, 3 recovery conditions and 4 evidence types.
- **[Current rule] Declared, not exercised.** A profile declares planned coverage. No situation has been exercised against a system, no vendor endpoint is named, and no market may be described as covered by evidence.
- **[Current rule] Environment faults are not agent failures.** An unreachable service, an evaluator defect or a broken starting state makes an attempt ineligible. It is discarded with a reason before scoring, never counted as a failure.

## Completion, handoff, safety and final-state evaluation

The hardest definitional problem in the method: deciding when an errand is actually done, in services that finish asynchronously and off the agent's own surface.

- **[Current rule] Final state, not transcript.** Success is a declared final state that an evaluator can check, expressed as positive obligations the state must satisfy together with invariants and prohibited-state checks the run must not have violated. (see: swe-bench, webarena, osworld)
- **[Current rule] A correct stop can be completion.** Stopping and handing back at an OTP, a carrier identity check, a government identity check, a QR or device payment, or an account-holder approval is valid completion when the task contract declares that boundary as the completion point.
- **[Current rule] A successful call is not proof.** For asynchronous or divergent-state journeys, an accepted API or tool response does not establish the final state. The run must produce an authoritative receipt, a post-action readback, or a handoff record. (see: api-bank, gaia, tau-bench)
- **[Current rule] Non-idempotent retries are guarded.** Where a repeat could create a second order, a second booking or a second charge, the task contract requires an idempotency record. A duplicate created by a retry is a failure of the attempt, not a second chance at it.
- **[Current rule] Five diagnostic outcomes, one binary factor.** Every attempt is classified as confirmed success, unverified completion, partial progress, recoverable failure or critical failure. The classification explains the outcome; the accuracy factor still admits no partial credit.
- **[Current rule] Critical safety blocks permanently.** A critical safety event permanently blocks publication of the cell it occurred in. The blocked cell is not dropped from aggregation: every family and market figure above it is withheld. There is no re-run that erases it and no appeal that removes it from the record.

## Per-task scoring and aggregation

The three raw axes are the record. A product may be derived per eligible validated attempt for audit, but it is not a headline result or ranking key until references, repeat runs and uncertainty are calibrated in the pilot. The product is MICA's own normative choice.

- **[MICA policy] Zero veto.** Because accuracy is binary and the form is multiplicative, any outcome other than a confirmed success sends the whole product to zero. Speed and cost cannot compensate for not finishing, and that non-compensatory behaviour is deliberate. (see: oecd-jrc-composite)
- **[MICA policy] Scaling assumptions.** Multiplying normalized components assumes the ratio scales are commensurable and that a proportional loss on one factor is worth the same as the equivalent loss on another. MICA has not established that this holds and does not present the product as a formal utility. (see: keeney-raiffa, fleming-wallace)
- **[MICA policy] No external endorsement.** No cited work derives or endorses this multiplication. The composite-indicator and multi-objective literature is cited here to state the conditions MICA has not met, not to lend the formula authority. (see: oecd-jrc-composite, keeney-raiffa, fleming-wallace)
- **[Current rule] Sensitivity and audit requirement.** No official figure may be published without a sensitivity analysis over the scoring policy, showing how a ranking moves under alternative normalizations and aggregations, and task-level results are published so the analysis can be redone independently. (see: oecd-jrc-composite, benchmark-lottery)
- **[Current rule] Zero cost is not a perfect score.** The cost component requires a strictly positive metered evaluation cost. Zero or unmetered cost is recorded as not measured and receives no score; it is never treated as perfect efficiency.
- **[External evidence] Macro and micro weighting.** Averaging over tasks and averaging over markets are different weightings with different meanings. MICA applies the distinction explicitly at the family, market and cross-market level rather than letting a denominator decide it silently. (see: sokolova-lapalme)

## Model routing, memory and tool evidence

A system may use whichever models it judges best, including several within one attempt. What it may not do is decline to show what it used.

- **[Current rule] Routing is part of the evaluated system.** Choosing a model per task, and calling several models within one attempt, are legitimate design decisions that carry no penalty of their own. They are scored only through the accuracy, speed and evaluation cost the whole system delivered.
- **[Current rule] Required invocation evidence.** Every model invocation records provider, model, version, purpose, tokens, cost, latency and its order within the attempt. An attempt with an unaccounted invocation is not scored. (see: bfcl, api-bank)
- **[Current rule] Memory is in scope.** Carrying persona facts, prior constraints and in-task state without re-asking the user or contradicting itself is part of the system under evaluation, and memory declared but not exercised is recorded as such.
- **[External evidence] Correct non-use counts.** Deciding correctly that no tool is needed, or that a tool must not be called, is measurable behaviour and is recorded rather than ignored. (see: bfcl)

## Tracks, reliability, uncertainty and missing data

How a result would be produced, how confident anyone could be in it, and what MICA writes when there is nothing to write.

- **[Current rule] Three tracks.** Simulator runs against deterministic local replicas; live shadow runs stop at the confirmation boundary against real services; limited verified live covers a small number of end-to-end runs on MICA-held test accounts. All three are design intent, not an implemented harness.
- **[Current rule] No live integration exists.** There is no vendor integration, no live account and no measured run on any track. The track vocabulary describes what a future run would be, not what has happened.
- **[External evidence] Repetition before comparison.** Single-run point estimates mislead, especially on small cells near 0 or 1. Repeated attempts, interval estimates and reliability across repeats are required before any comparison between systems is published. (see: agarwal-statistical, tau-bench)
- **[Open question] Uncertainty for binary hierarchical outcomes.** MICA's outcomes are binary and nested inside task, family and market. Which interval and resampling procedure is correct for a product score over that structure is not settled, and the choice will be published with its justification before any interval appears. (see: agarwal-statistical)
- **[Current rule] Absence is not zero.** A task that was not attempted, was not eligible, or has no pre-registered reference has no score and remains listed with a reason. If the pre-registered canonical task set is incomplete, the family and market aggregate is withheld rather than computed over a shrinking denominator. A missing market likewise withholds any cross-market figure.
- **[Current rule] Compatible secondary reporting.** MICA reports raw success rates, latency distributions and metered evaluation cost as the primary record, with reliability across repeats when available. The per-attempt product is a derived audit view of that record during the pilot, not a competing headline score. (see: helm, agarwal-statistical)

## Evidence lineage, reproducibility and privacy

Every published figure must be walkable back to the thing that produced it, without publishing anything that identifies a person or exposes an account.

- **[Current rule] The lineage chain.** Edition, then suite and task contract, then country and local situation, then system snapshot, then attempt or run, then model and tool calls, then evaluator evidence and final-state readback, then the aggregate. A figure that cannot be walked back along this chain is not published.
- **[Current rule] Redacted public traces.** Public traces are redacted. Originals are retained under access control, so an independent reviewer can be given the unredacted record without it becoming a public artifact. (see: pineau-reproducibility)
- **[Current rule] Synthetic personas only.** Runs use synthetic personas and MICA-held test accounts. No irreversible action is ever performed against an account belonging to a real person.
- **[External evidence] Reproducibility record.** Each published result carries what was run, on which snapshot, how many times, under which track and conditions, and what was reported, following established reproducibility reporting practice. (see: pineau-reproducibility, helm)
- **[Current rule] Scores are derived, not stored.** The raw outcome, latency and evaluation cost are the record. Any score is computed from them on request, so a published figure cannot drift away from the evidence it claims to summarize.

## Public set, holdout, contamination and maintenance

A benchmark that is fully public becomes a training target. One that is fully private cannot be criticized. MICA keeps both halves and states which is which.

- **[Current rule] Separated sets.** Public candidates and the private holdout are separate sets, and the separation is enforced at export time rather than trusted to editorial care. A holdout item is never reachable from a public artifact.
- **[Current rule] Item provenance.** Each item records when it was created, when and where it was exposed, its version and its provenance, so exposure can be reasoned about instead of guessed at. (see: magar-schwartz, gpt3-contamination)
- **[Current rule] Stable core, rotating challenge.** The intended shape is a stable longitudinal core that supports comparison over time, plus a rotating challenge and holdout portion that limits how long exposure stays valuable. It is a design, and it is labelled planned until it is implemented. (see: livebench, dynabench, gaia)
- **[Current rule] No claim of contamination freedom.** MICA will not claim a set is uncontaminated. It controls exposure, records provenance, refreshes items and reports residual risk, and distinguishes exposure from memorization and from score exploitation. (see: magar-schwartz, livebench, gpt3-contamination)
- **[Open question] Suite dependence.** Which tasks are in the suite changes which system looks best. MICA publishes task-level results and requires sensitivity analysis, but how to choose a suite that is not itself a hidden preference is unresolved. (see: benchmark-lottery)

## Governance, independence and disclosure

An index that measures systems whose makers can fund it has to say who decides what, and where that separation is not yet real.

- **[Current rule] Neutrality requirement.** MICA must be neutral. vooy may be an originator and may be measured as a challenger system, and holds no privileged position as benchmark owner in the normative public design.
- **[Current rule] What must be disclosed.** The operator, who holds decision rights, funding and its terms, paid task-author relationships, vendor relationships, conflicts of interest, the publication process, and the corrections and appeals procedure. (see: nist-ai-rmf)
- **[Open question] Structural independence.** Decision rights over method changes, verification outcomes and publication must rest with a body no evaluated party controls. No such body has been constituted, so this is an open risk and not a solved claim.
- **[Current rule] No paid placement.** MICA does not accept payment for placement, for inclusion, or for the timing of a result, and does not accept a submitter's own score or weighting in place of a measurement.

## How to challenge or amend the method

This wiki is written to be argued with. A challenge that names a section and carries evidence is the fastest way to change what MICA does.

- **What counts as a challenge.** A specific claim in this wiki, named by its section id, with the reason it is wrong and what would replace it. A preference is not a challenge; a counter-example, a citation or a reproducible discrepancy is.
- **Evidence a challenge must carry.** For a method claim, a citation or a worked counter-example. For a locality claim, the market, the channel and the convention being misdescribed. For a scoring claim, the inputs and the resulting figure so the disagreement is reproducible.
- **Where to send it.** Through the submission route, which states the evidence requirements MICA applies to anything it receives. (see `/submit`)
- **Who decides.** The governance page states who holds decision rights today and which of those rights are not yet independently held. MICA does not describe an authority it does not have. (see `/about/governance`)
- **How the method changes.** The method is versioned with the edition. A change ships as a new edition of this wiki and of the generated Markdown pack, both of which state the current rule in full so a reader never has to reconstruct it.
- **Corrections.** A figure MICA cannot re-derive from the record it holds is withdrawn, not annotated. A correction is published with the same prominence as the claim it corrects.

## Limitations and open research questions

The parts of the method MICA does not consider settled. A benchmark that hides these is advertising.

- **[Open question] Is the product the right form.** The multiplicative form is a policy choice with strong non-compensatory consequences. Whether a different aggregation would better reflect what a buyer values is genuinely open, and the sensitivity requirement exists so the question stays answerable. (see: oecd-jrc-composite, keeney-raiffa)
- **[Open question] The limit of the priority claim.** Worldwide priority cannot be proven. A finite literature review establishes what MICA found, not what exists, so the world-first claim is dated to its 8 August 2026 cutoff, scoped to a stated conjunction of conditions, and open to challenge through the amendment process on the same evidence terms as any other claim here. If a prior public initiative meeting the complete conjunction is identified, the claim is revised or withdrawn rather than defended. (see: webarena, webvoyager, webshop, androidworld, appworld, tau-bench)
- **[Open question] How references should be set.** Speed and cost references are pre-registered per task version and never re-derived from the cohort being scored. How to set them so they are demanding but attainable, without anchoring on any one system, is unresolved.
- **[Open question] Does simulator behaviour transfer.** Deterministic replicas are reproducible but synthetic, and live services change under the runner. How much simulator performance predicts live performance is exactly what the shadow track is meant to test, and it has not been tested.
- **[Open question] Uneven market coverage.** Markets differ in how much of a journey is reachable at all. A cross-market figure computed over unevenly reachable markets may compare accessibility rather than capability, and MICA withholds such figures rather than estimating them.
- **[Open question] How deep locality must go.** A country is not homogeneous, and a task assumes a region, a channel and a register. How finely a market edition must be stratified before it is fair to call a result a result for that market is not settled. (see: blend, mega)
- **[Open question] Cost moves independently of behaviour.** Evaluation cost depends on provider pricing on the snapshot date, which moves for reasons that have nothing to do with the system. Physical usage quantities and dated price schedules are preserved so cost can be recomputed, but the volatility is real.

## Current policy register

The current statements MICA is prepared to be held to. Every line is present tense: this register is not a log, and it records no earlier value.

- **Project definition.** MICA, the Multinational Index of Consumer Agents, is a benchmark for complete versioned consumer-agent systems. Its canonical public domain is micabench.com.
- **Market scope.** The index covers 6 markets: KR, JP, SG, TW, AE, TH. This is the scope of the index, and no other market is in it.
- **Task catalogue.** 10 task families hold 100 canonical definitions. They are provisional public-set candidates: not executable validated scenarios, not measured results, and not a holdout.
- **Derived per-attempt audit score.** Raw accuracy, latency and metered evaluation cost are the record and stay separately disclosed. MICA may derive 100 × accuracy × speed × cost for each eligible validated attempt, but during the pilot the product is an audit value, not a headline result or ranking key. Family and market figures are withheld unless the pre-registered canonical task set is complete. The product is a MICA policy, not a result derived from external literature.
- **Public candidates and holdout.** The public candidate set and the private holdout set are separate, and separation is enforced at export. Creation, exposure, version and provenance are recorded per item.
- **Market integration profiles.** Each of the 6 market editions declares representative surfaces, authorization boundaries, completion semantics, recovery conditions and evidence requirements. These declare planned coverage, never measured coverage.
- **Neutral governance intent.** MICA must be neutral. vooy may act as originator and as a challenger system, and holds no privileged position as benchmark owner in the normative public design. Operator, decision rights, funding, paid task-author relationships, vendor relationships and conflicts are disclosed. Structural independence is an open requirement, not a solved claim.

## Research references by methodological contribution

Grouped by what MICA takes from each, with what it deliberately does not take. Citation is not endorsement: no work listed here has reviewed or approved MICA.

### Agent and web environments

Where MICA takes its execution-based, final-state notion of a task from, and where those environments stop short of a localized consumer market.

- [WebArena](https://arxiv.org/abs/2307.13854) — A Realistic Web Environment for Building Autonomous Agents. ICLR 2024.
  - Borrowed: Realistic resettable websites with task-specific final-state validators, rather than a judge scoring a transcript.
  - Not adopted: Its site set is generic and English-first. MICA extends the same validator idea to localized consumer markets and to surfaces that have no browsable equivalent.
- [VisualWebArena](https://aclanthology.org/2024.acl-long.50/) — Evaluating Multimodal Agents on Realistic Visual Web Tasks. ACL 2024.
  - Borrowed: Rendered-state tasks and visual evidence: what the user can actually see is part of the task, not an implementation detail.
  - Not adopted: MICA does not assume DOM-only access anywhere in the method, so a system that only reads markup is not privileged by the design.
- [WorkArena](https://proceedings.mlr.press/v235/drouin24a.html) — How Capable Are Web Agents at Solving Common Knowledge Work Tasks?. ICML 2024.
  - Borrowed: Parameterized task templates validated against backend state, so one task definition yields many checkable instances.
  - Not adopted: Its domain is enterprise knowledge work on one platform. MICA's unit is a household consumer journey across independent local services.
- [OSWorld](https://arxiv.org/abs/2404.07972) — Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. NeurIPS 2024.
  - Borrowed: Snapshot and reset discipline, and execution-based evaluators that read computer state instead of trusting a claim of success.
  - Not adopted: A resettable desktop cannot reset a third-party merchant, a bank or a carrier. MICA's parity conditions do that work instead.
- [AndroidWorld](https://arxiv.org/abs/2405.14573) — A Dynamic Benchmarking Environment for Autonomous Agents. arXiv 2024 / ICLR 2025.
  - Borrowed: Parameterized mobile tasks with executable app-state checks, which is the closest published analogue to app-enclosed consumer journeys.
  - Not adopted: Its apps are installed and controlled by the harness. MICA's markets include signed-in channels the harness will never own.
- [Mind2Web](https://arxiv.org/abs/2306.06070) — Towards a Generalist Agent for the Web. NeurIPS 2023.
  - Borrowed: Breadth: a task taxonomy spanning many websites and domains, and an explicit test of generalization to unseen sites.
  - Not adopted: Offline action matching against recorded traces is not proof of final state. MICA does not accept step agreement as success.
- [WebLINX](https://proceedings.mlr.press/v235/lu24d.html) — Real-World Website Navigation with Multi-Turn Dialogue. ICML 2024.
  - Borrowed: Multi-turn conversational traces and website-disjoint splits, which is how MICA thinks about a user who changes their mind mid-journey.
  - Not adopted: Offline trace metrics stay diagnostic in MICA. They never become the primary success criterion for a task.
- [AssistantBench](https://arxiv.org/abs/2407.15711) — Can Web Agents Solve Realistic and Time-Consuming Tasks?. 2024.
  - Borrowed: Long-horizon realistic tasks, and the practice of reporting the effort a task actually cost alongside whether it succeeded.
  - Not adopted: Its outcomes are largely information answers. MICA's unit is a persistent state change in a real consumer service.
- [WebShop](https://arxiv.org/abs/2207.01206) — Towards Scalable Real-World Web Interaction with Grounded Language Agents. NeurIPS 2022.
  - Borrowed: Scalable consumer shopping goals expressed as constraint satisfaction over an instruction, which MICA reuses in its shopping family contracts.
  - Not adopted: A single simulated store is narrower than a market. MICA's shopping tasks cross payment, identity and delivery rails that a single store hides.
- [WebVoyager](https://arxiv.org/abs/2401.13919) — Building an End-to-End Web Agent with Large Multimodal Models. ACL 2024 / arXiv v1 25 January 2024.
  - Borrowed: Operation on live websites rather than a frozen replica: 643 tasks across 15 real sites, which is the published precedent for evaluating against services the harness does not own.
  - Not adopted: There is no systematic per-country localized suite, and success is judged in part by a GPT-4V reading of screenshots and agent output rather than by authoritative market-side completion. MICA requires a receipt, readback or handoff record instead.
- [AppWorld](https://arxiv.org/abs/2407.18901) — A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. ACL 2024 / arXiv v1 26 July 2024.
  - Borrowed: Everyday consumer domains spanning multiple interacting apps and people, checked by state-based tests rather than by output matching.
  - Not adopted: Its apps are a controllable simulated world authored for the benchmark, not 6 independently localized real-life markets with their own payment, identity and messaging rails.

### Tool use and stateful interaction

Work on tool calls, policies, backend state and repeated reliability, which shapes what MICA demands as invocation evidence.

- [tau-bench](https://arxiv.org/abs/2406.12045) — A Benchmark for Tool-Agent-User Interaction in Real-World Domains. 2024.
  - Borrowed: Stateful user, agent and tool interaction judged against domain policy and backend state, plus repeated-trial reliability rather than one lucky run.
  - Not adopted: It covers two simulated service domains, retail and airline, in a single language. They are not 6 country-localized real consumer markets, and MICA's policies are the authorization conventions those markets actually apply.
- [API-Bank](https://aclanthology.org/2023.emnlp-main.187/) — A Comprehensive Benchmark for Tool-Augmented LLMs. EMNLP 2023.
  - Borrowed: The decomposition of tool use into deciding a tool is needed, retrieving it, forming arguments, executing and reading the response.
  - Not adopted: A well-formed call is not a completed errand. MICA scores the final state, and uses call diagnostics only to explain it.
- [Berkeley Function-Calling Leaderboard](https://gorilla.cs.berkeley.edu/leaderboard.html) — Structured function-call evaluation, including non-use, multi-call and multi-turn categories. Evolving leaderboard; cite with the access date.
  - Borrowed: Call normalization before comparison, and the insight that correctly declining to call a tool is a measurable behaviour.
  - Not adopted: It is a live, versionless surface. MICA cites it as a dated evolving resource and never treats a snapshot of it as a fixed standard.
- [AgentBench](https://arxiv.org/abs/2308.03688) — Evaluating LLMs as Agents. ICLR 2024.
  - Borrowed: Heterogeneous interactive environments reported per environment, so a strength in one setting cannot be laundered into a general claim.
  - Not adopted: MICA does not aggregate unlike rewards into one figure. Family and market means are taken only over commensurable per-task scores.
- [GAIA](https://arxiv.org/abs/2311.12983) — A Benchmark for General AI Assistants. ICLR 2024.
  - Borrowed: Real-world multimodal tool tasks with unambiguous answers, and a hidden split held back from the public set.
  - Not adopted: A correct final answer is not a persistent state change. MICA requires an authoritative receipt or readback, not a string match.
- [SWE-bench](https://arxiv.org/abs/2310.06770) — Can Language Models Resolve Real-World GitHub Issues?. ICLR 2024.
  - Borrowed: Success as positive obligations plus regression and invariant checks, with a per-instance artifact that makes a claim auditable.
  - Not adopted: Its invariants are test suites the harness owns. MICA has to express the same idea as prohibited-state checks on services it does not own.

### Measurement, composites and uncertainty

The literature MICA uses to criticize its own product score, not to justify it. Nothing here endorses multiplying heterogeneous factors.

- [HELM](https://arxiv.org/abs/2211.09110) — Holistic Evaluation of Language Models. TMLR 2023.
  - Borrowed: The separation of scenario, system, metric and run, and the discipline of multidimensional disclosure instead of one headline number.
  - Not adopted: HELM does not justify a single universal product score. MICA's per-task product is its own decision and is defended as such.
- [MLPerf Inference](https://doi.org/10.1109/ISCA45697.2020.00045) — MLPerf Inference Benchmark. ISCA 2020.
  - Borrowed: Latency is comparable only under a fixed workload, a fixed quality target and a declared system boundary. MICA fixes all three before timing anything.
  - Not adopted: Its workloads are deterministic and hardware-bounded. A consumer market changes under the runner, so MICA reports speed with the conditions attached.
- [OECD / JRC](https://doi.org/10.1787/9789264043466-en) — Handbook on Constructing Composite Indicators: Methodology and User Guide. 2008.
  - Borrowed: Normalization, weighting and compensability are substantive decisions that must be documented and tested, not formatting choices.
  - Not adopted: MICA uses this handbook to criticize its own product and to require sensitivity analysis. It is not an endorsement of MICA's formula.
- [Keeney & Raiffa](https://doi.org/10.1017/CBO9781139174084) — Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press edition.
  - Borrowed: The conditions under which a multiplicative form is a legitimate utility function: stated preference structure and specific independence assumptions.
  - Not adopted: MICA has not established those assumptions, so its per-task product is a scoring policy and is not claimed to be a formal utility.
- [Fleming & Wallace](https://doi.org/10.1145/5666.5673) — How Not to Lie with Statistics: The Correct Way to Summarize Benchmark Results. CACM 1986.
  - Borrowed: Geometric means are the right summary for commensurable positive ratios, which is exactly the shape of a normalized speed or cost component.
  - Not adopted: It does not license multiplying heterogeneous metrics in general, and MICA does not cite it as if it did.
- [Sokolova & Lapalme](https://doi.org/10.1016/j.ipm.2009.03.002) — A Systematic Analysis of Performance Measures for Classification Tasks. Information Processing & Management, 2009.
  - Borrowed: The macro versus micro distinction: whether you average over items or over groups is a weighting decision with visible consequences.
  - Not adopted: The original setting is classification metrics. MICA applies only the weighting concept, to tasks within a family and families within a market.
- [Agarwal et al.](https://arxiv.org/abs/2108.13264) — Deep Reinforcement Learning at the Edge of the Statistical Precipice. NeurIPS 2021.
  - Borrowed: Few-run point estimates mislead. Report repeated runs, uncertainty intervals and performance distributions rather than a single mean.
  - Not adopted: Its estimators assume continuous returns. MICA's outcomes are binary and hierarchical, so the adaptation has to be justified, not copied.

### Consumer-platform execution controls

Official platform documentation, cited as evidence of the conditions under which a consumer action can be executed in production. None of these vendors has reviewed, approved or endorsed MICA.

- [Kakao Developers](https://developers.kakao.com/docs/latest/en/getting-started/app) — App. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes that use of the platform begins with a registered application holding app keys and per-product configuration, so the registered app is the unit access is granted against.
  - Not adopted: It does not establish that registration grants every capability. What a registered app may actually do is set per product and per scope, so MICA does not infer execution rights from registration.
- [Kakao Developers](https://developers.kakao.com/docs/latest/en/message/rest-api) — Kakao Talk Message · REST API. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes that sending a message depends on a user access token, a consented scope, a supported message template and a permitted recipient relationship, which is a materially different condition from read-oriented local search.
  - Not adopted: It does not establish that messaging is unavailable, and MICA does not read it that way. It establishes that the send path is conditioned, which is why declared coverage is never treated as executable coverage.
- [LINE Developers](https://developers.line.biz/en/docs/line-developers-console/creating-channel/) — Creating a channel. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes a provider and channel structure as the control plane: a channel is created for a specific product and carries its own credentials.
  - Not adopted: Creating a channel is not blanket product approval, and this page does not say it is. Individual products keep their own eligibility and review conditions.
- [LINE Developers](https://developers.line.biz/en/docs/messaging-api/sending-messages/) — Sending messages. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes that a send depends on an Official Account channel and its access token, on the recipient relationship, and on which send method applies, rather than on a generic outbound call.
  - Not adopted: It does not establish anything about non-messaging surfaces, and MICA does not generalize from it to payment, commerce or account operations on the same platform.
- [LINE Developers](https://developers.line.biz/en/docs/line-mini-app/develop/development-flow/) — LINE MINI App development flow. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes a lifecycle in which development and testing precede a review step before release, so reaching production is a gated transition rather than a deployment.
  - Not adopted: It does not establish uniform worldwide availability. Regional availability and per-region conditions can differ, so MICA records executability per market rather than per platform.
- [WeChat Mini Program](https://developers.weixin.qq.com/miniprogram/en/dev/framework/quickstart/release.html) — Release. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation establishes an account-controlled runtime bound to an AppID, where a build is uploaded, submitted for review and only then released to users.
  - Not adopted: The English page is not a complete statement of current conditions. It can lag the Chinese documentation and the admin console, so MICA cites it for the shape of the control and verifies specifics per market edition.
- [Grab Developer](https://developer.grab.com/docs/) — Grab Developer Documentation. Official documentation · accessed 8 August 2026.
  - Borrowed: The documentation root establishes that access is organized around a developer application and product-specific onboarding, with some production capabilities conditioned by country or partner status.
  - Not adopted: It does not establish which specific capability is open in which market on a given date. MICA cites the documentation root rather than a deep link because product pages move, and it verifies specifics per market edition.

### Contamination, reproducibility, localization and governance

How a benchmark stays honest over time: exposure control, documentation, per-language reporting and documented ownership.

- [Brown et al. (GPT-3)](https://arxiv.org/abs/2005.14165) — Language Models are Few-Shot Learners. NeurIPS 2020.
  - Borrowed: Benchmark overlap audits and clean-subset comparison as a routine, published part of a result rather than an afterthought.
  - Not adopted: Overlap auditing is not a proof of cleanliness. MICA states residual contamination risk rather than declaring a set uncontaminated.
- [Magar & Schwartz](https://aclanthology.org/2022.acl-long.18/) — Data Contamination: From Memorization to Exploitation. ACL 2022.
  - Borrowed: Exposure, memorization and score exploitation are three different things, and only the third necessarily corrupts a comparison.
  - Not adopted: MICA cannot measure exploitation directly today, so it controls exposure and records provenance instead of claiming exploitation is absent.
- [Dynabench](https://aclanthology.org/2021.naacl-main.324/) — Rethinking Benchmarking in NLP. NAACL 2021.
  - Borrowed: Human and model in the loop challenge collection, which is how MICA intends to keep a rotating challenge set adversarial.
  - Not adopted: Adversarial collection skews toward edge cases. MICA keeps a stable longitudinal core so the index does not drift into a corner-case suite.
- [LiveBench](https://arxiv.org/abs/2406.19314) — A Challenging, Contamination-Free LLM Benchmark. 2024 / ICLR 2025.
  - Borrowed: Regular refresh from recent material with objectively verifiable answers, which limits how long an exposed item stays worth exploiting.
  - Not adopted: MICA will not claim contamination freedom. Refresh reduces risk; it does not eliminate it, and the wiki says so.
- [Pineau et al.](https://jmlr.org/papers/v22/20-303.html) — Improving Reproducibility in Machine Learning Research (Reproducibility Program Report). JMLR 2021.
  - Borrowed: A reproducibility checklist attached to every published result: what was run, on what, how many times, and what was reported.
  - Not adopted: Full artifact release is impossible where a trace contains account state. MICA publishes redacted traces and keeps originals access-controlled.
- [Datasheets for Datasets](https://doi.org/10.1145/3458723) — Documentation of motivation, composition, collection, uses, distribution and maintenance. CACM 2021.
  - Borrowed: The documentation frame MICA applies to each edition of the task catalogue, one datasheet section per axis.
  - Not adopted: A datasheet describes a dataset. MICA's catalogue is a set of executable contracts, so it also documents environment and authorization conditions.
- [MEGA](https://aclanthology.org/2023.emnlp-main.258/) — Multilingual Evaluation of Generative AI. EMNLP 2023.
  - Borrowed: Per-language reporting as a requirement, because an average across languages hides exactly the market where a system fails.
  - Not adopted: Multilingual coverage is not cultural localization. MICA treats language as one input to locality, not as locality itself.
- [BLEnD](https://arxiv.org/abs/2406.09948) — A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages. NeurIPS 2024.
  - Borrowed: Everyday cultural knowledge collected locally rather than inferred, which is the standard MICA holds its task authors to.
  - Not adopted: A country is not homogeneous. MICA records the region, channel and register a task assumes rather than labelling it with a flag.
- [M3Exam](https://arxiv.org/abs/2306.05179) — A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models. AAAI 2024.
  - Borrowed: Locally sourced material behaves differently from translated English material, which is why MICA requires local authorship rather than translation.
  - Not adopted: Exam items have fixed answers. A consumer errand's correct outcome depends on live local state, so MICA validates final state instead.
- [Data Statements for NLP](https://aclanthology.org/Q18-1041/) — Toward Mitigating System Bias and Enabling Better Science. TACL 2018.
  - Borrowed: Documenting language variety, collection context and the situation of the people who authored and reviewed the material.
  - Not adopted: MICA records reviewer context at the level of role and market competence only. No individual is identifiable from a published statement.
- [NIST AI RMF 1.0](https://doi.org/10.6028/NIST.AI.100-1) — Artificial Intelligence Risk Management Framework. NIST, 2023.
  - Borrowed: The GOVERN, MAP, MEASURE and MANAGE structure, and its demand for documented ownership, validity criteria and ongoing monitoring.
  - Not adopted: MICA claims no NIST alignment. A conformance claim would require a published mapping, and MICA has not produced one.
- [The Benchmark Lottery](https://arxiv.org/abs/2107.07002) — How benchmark, task and metric choice can change the conclusion. 2021.
  - Borrowed: The warning that a ranking can be an artifact of the suite. MICA publishes task-level results so a conclusion can be recomputed differently.
  - Not adopted: The lesson is not that ranking is impossible. It is that a ranking without sensitivity analysis is not evidence, which MICA accepts as a requirement.

## Publication and interface references

Product and editorial influences on how MICA publishes. They are kept separate from the scientific precedents because an interface convention is not evidence about measurement.

- [Inspect](https://inspect.aisi.org.uk/) — Evaluation framework with per-sample trace drilldown. The shape of a trace drilldown: from an aggregate straight to the sample and the calls behind it. Internal observability is not independent benchmark evidence, and MICA does not present it as such.
- [Braintrust](https://www.braintrust.dev/) — Experiment and evaluation tooling for LLM products. Experiment records that keep the input, the output and the grader together. Useful as a product pattern, not as a source of external verification.
- [RTINGS](https://www.rtings.com/) — Product testing with published methodology versions. Published test definitions with explicit version histories, and raw measured values shown beside any derived rating.
- [Wirecutter](https://www.nytimes.com/wirecutter/) — Conditional recommendations with stated scope. Recommendations written as conditional on a use case, with the scope of testing stated rather than implied.
- [Ookla Speedtest Global Index](https://www.speedtest.net/global-index) — Country-first entry into a measured index. Country-first navigation into a measurement index, and a visible statement of what the median figure does and does not represent.
- [Transparency International CPI](https://www.transparency.org/en/cpi) — Composite country index with published source and limits. A composite country index that publishes its sources, its construction and its limits together, and warns against year-on-year over-reading.
- [Freedom House](https://freedomhouse.org/report/freedom-world) — Freedom in the World: scored country reports with method notes. Country reports where the narrative and the score are published side by side, so the number can be argued with using the report's own text.
- [World Justice Project](https://worldjusticeproject.org/rule-of-law-index) — Rule of Law Index: sub-factor disclosure beneath a headline. Sub-factor scores always reachable beneath a headline figure, so a country's position can be decomposed rather than taken on trust.
- [WCAG 2.2](https://www.w3.org/TR/WCAG22/) — Web Content Accessibility Guidelines 2.2. The conformance target for this site: text alternatives to colour, focus visibility, reflow without horizontal scrolling and meaningful link text.
- [W3C WAI Tables Tutorial](https://www.w3.org/WAI/tutorials/tables/) — Accessible data table patterns. Header association, captions and scope for any data table MICA eventually publishes results into.
- [GOV.UK Content Design](https://www.gov.uk/guidance/content-design) — Guidance on writing for a public audience. Plain, front-loaded sentences and a refusal of promotional register in factual public content.
- [Our World in Data](https://ourworldindata.org/) — Public data communication with sources and definitions attached. Every chart carries its definition, its source and its coverage gaps in reach of the reader, not in a separate appendix.
- [FT Visual Vocabulary](https://github.com/Financial-Times/chart-doctor/tree/main/visual-vocabulary) — Chart-type selection by analytical intent. Choosing a chart from the relationship being shown rather than from what looks impressive, which is why MICA has no dashboard.

## Glossary

The terms this wiki uses in a specific sense. Where a word could mean two things, this is the one MICA means.

- **System snapshot.** A complete consumer-agent system frozen at a stated version, including scaffold, routing, memory, tools, permissions, localization and recovery behaviour. The unit MICA evaluates.
- **Task contract.** The declared final state of a task, its confirmation boundary, its termination class and the evidence a run must produce for the outcome to count.
- **Confirmation boundary.** The point past which an action is irreversible or requires an authorization the agent must not perform on the user's behalf. Stopping there correctly is a defined outcome, not a failure.
- **Eligible attempt.** An attempt that passed screening: the environment was reachable, the persona and accounts were in the declared starting state, and no evaluator-side fault interrupted it.
- **Confirmed success.** The declared final state was reached and evidenced by an authoritative receipt, readback or handoff record. The only outcome that scores 1 on the accuracy factor.
- **Unverified completion.** The system reports it finished, but the required authoritative evidence is absent. Diagnostically distinct from failure; scored 0 on accuracy all the same.
- **Evaluation execution cost.** Model, routing, retry, subagent, browser, search, maps and paid tool spend for running the attempt, in USD. Never the purchase price of goods or services the agent handled.
- **Integration situation.** One representative local execution condition in a market edition, carrying its surfaces, authorization boundary, completion semantics, recovery conditions and evidence requirements.
- **Public candidate.** A published task definition available for inspection and criticism. Design material for the method, not an item of the final executable benchmark.
- **Holdout.** An item withheld from publication so that exposure cannot be converted into score. Never exported, and never reachable from a public artifact.
- **Evidence lineage.** The unbroken chain from edition to suite and task contract, market situation, system snapshot, attempt, model and tool calls, evaluator evidence and final-state readback, up to the aggregate.
- **Fail closed.** When a required condition is unset or unmet, the gate refuses to publish rather than substituting a default. Absence never becomes a passing value.

## Current operational contract

### Publication rules

- **Minimum eligible runs.** Not set. MICA has not agreed a minimum sample size, so no cell can be judged against one. The gate refuses rather than guessing a number.
- **Minimum coverage.** Not set. Until MICA agrees what share of a market's canonical tasks must be attempted, coverage cannot be a pass condition.
- **Thresholds.** Publication thresholds are not set for this edition: MICA has not yet agreed runs per task, a minimum cell sample size, task coverage, or a maximum ineligible discard rate, so no result can be marked publication eligible.
- **Critical safety events.** Any critical safety event permanently blocks publication of the cell. The blocked cell is not removed from its aggregate: family and country figures above it are withheld. There is no re-run to erase the event.
- **Verification.** Independent rerun only. Provisional and self-reported results are displayed for transparency and never published as official.
- **Data status.** Demo and preview data can never be publication eligible. This is enforced in the schema layer, which throws at build time if a demo record claims otherwise.

### The three outcome axes

- **Accuracy · share of eligible runs reaching the confirmed final state.** Did the system reach the task's declared final state, inside the confirmation boundary, without an unrecoverable error?
- **Speed · wall-clock seconds, successful eligible runs only.** How long a successful run took end to end. Failed runs are excluded so that fast failure never reads as fast success.
- **Cost · currency per successful eligible run.** Total eligible attempt cost divided by successful eligible runs. Cost of failure is charged to the successes it took to get there.

### Three raw axes first, one derived audit value

- **Per-task final score.** 100 × accuracy × speed × cost Each factor is a normalized component between 0 and 1, not a raw second and not a raw dollar. The product is therefore bounded by 0 and 100.
- **Accuracy component.** 1 for a confirmed success, 0 for every other outcome. No partial credit, so a faster or cheaper failure still scores zero: the product runs through that zero.
- **Speed component.** min(1, speed reference ÷ observed seconds). The reference is pre-registered per executable validated task version and is never re-derived from the cohort being scored, so a slow field cannot make a slow system look fast. Beating it is capped at 1.
- **Cost component.** min(1, cost reference ÷ metered evaluation cost in USD). Zero or unmetered cost is recorded as not measured rather than awarded a perfect component.
- **Family and country scores.** The arithmetic mean of eligible attempt-derived task scores in that family or market. It is withheld unless the pre-registered canonical task set is complete. Exclusions remain listed rather than silently shrinking the denominator.
- **Cross-market, cross-category overall.** Not defined. A single overall figure would need a weighting across markets and categories that MICA has not agreed, so none is published and none is implied by the leaderboard structure.
- **Unscored tasks.** A task that was not attempted, was not eligible, or has no pre-registered reference has no score at all. It is excluded from the mean with a stated reason and is never counted as a zero.
- **Raw disclosure.** The raw success outcome, raw wall-clock latency and raw evaluation cost stay published in their own units beside every score, and remain auditable independently of it.
- **What cost means in the formula.** Evaluation execution cost: the model, tool and API spend of running the task, in USD. It is not the price of an item, a booking or any transaction value the agent handled, so no currency conversion enters the product.
- **Current status.** No system has been measured and no task carries a score. The formula is published as a draft so it can be challenged before it produces a number.

### Eligibility and denominators

- **eligible run.**  is an attempt that passed screening: the environment was reachable, the persona and its accounts were in the declared starting state, and no MICA-side fault interrupted the run. Ineligible attempts are discarded before scoring rather than counted as failures.
- **Accuracy.**  is successful eligible runs divided by eligible runs, reported with a 95% Wilson score interval. The Wilson interval is used instead of the normal approximation because MICA cells are small and often sit near 0 or 1, where the normal approximation misbehaves.
- **Speed.**  uses wall-clock seconds from successful eligible runs only, reported as p50 and p95. Failed runs are timed and kept with the run cell, but they are excluded from the reported percentiles deliberately: if they were included, a system that gives up quickly would read as fast. The population behind a speed figure is therefore the successes, and it is smaller than the eligible-run denominator behind accuracy.
- **Cost.**  is the total cost of all eligible attempts divided by the number of successful ones, in the market’s own currency. The cost of failure is charged to the successes it took to get there. With zero successes the value does not exist, and the table says “No successful task.” rather than showing a zero or an infinity.
- **country macro-average.** Cross-market figures use a : the mean of per-country values, computed only when every market is present. A missing market withholds the global figure instead of being treated as a zero.

### One system, many models

- **Per-task routing.** A system may send different tasks to different models and select whichever model it judges best for a given task.
- **Multiple invocations.** One task attempt may call several models. That is a legitimate design choice and carries no penalty of its own.
- **Invocation lineage.** Every invocation is disclosed as evidence: provider, model, version, purpose, tokens, cost, latency and order within the attempt.
- **What is scored.** The attempt, not the route. Routing shows up in the score only through the accuracy, speed and evaluation cost the whole system actually delivered.

### Seven diagnostic axes

- **Orchestration.** Decomposition, sequencing, retries, and the decision to stop. The largest observed source of spread between systems sharing a base model.
- **Model Routing.** Whether the system sends each sub-step to an appropriate model, and whether it degrades gracefully when a route is unavailable.
- **Memory.** Carrying persona facts, prior constraints, and in-task state across steps without re-asking the user or contradicting itself.
- **Tool / API Use.** Correct, minimal, and well-formed use of the tools the system declares, including handling of partial and error responses.
- **Localization.** Language, address and name formats, local payment and identity rails, holidays, and market-specific service conventions.
- **Safety.** Respecting the confirmation boundary, avoiding irreversible action without consent, and refusing out-of-scope credential use.
- **Recovery.** Behaviour after a failed step: detection, diagnosis, alternative route, or an honest stop with state left clean.

### Verification status and result tracks

- **Independent rerun · IND.** MICA re-executed the submitted system snapshot on MICA-controlled accounts and infrastructure, and holds the full evidence trace. On the publication track.
- **Provisional · PROV.** MICA observed a partial or supervised run, or holds evidence for only part of the claimed coverage. Reported, never published as official. Never published as official.
- **Self-reported · SELF.** Submitted by the system's operator with an evidence trace MICA has not reproduced. Displayed for transparency only. Never published as official.
- **Simulator.** Deterministic local replicas of common local service flows. Reproducible, cheap, and the only track where MICA can guarantee identical conditions across systems.
- **Live shadow.** The system acts against real services but stops at the confirmation boundary; no irreversible action is taken. Used to check that simulator behaviour transfers.
- **Limited verified live.** A small number of end-to-end runs on MICA-held test accounts where the operator of the service has been notified. Narrow by design.

### What each phrase means

- **No successful task..** There were eligible attempts, but none reached the declared final state, so speed and cost per success are undefined.
- **No coverage in this market..** The system has no eligible run cells for this market or slice at all. This is an absence of evidence, not a poor result.
- **Not measured.** The axis was not assessed for this snapshot. It is not a low reading.

### Measurement, interpretation, recommendation

- **Measured.** A value read directly from run records. No inference.
- **MICA interpretation.** MICA's reading of what the measured values mean. Contestable, and signed by the edition.
- **Recommendation.** Advice for builders or buyers. Never a claim of measurement.

### Representative local platforms are fixed before a run

- **Representative set.** For every market and task, MICA pre-registers a candidate set of platforms that ordinary consumers in that market commonly use, with evidence of local reach, current availability and task relevance.
- **Functional fit.** The selected platform must support the canonical task without changing its declared final state, constraints or confirmation boundary.
- **Access parity.** Every evaluated system receives the same selected platform, account tier, starting state, permissions and confirmation boundary.
- **Pre-registration and fallback.** The selected platform, version, snapshot date and ordered fallback list are fixed before systems run. A fallback is used only for a documented platform outage, never because it is easier for one system.
- **Evidence lineage.** Every published run cell names the platform actually used, its version and snapshot date, the selection evidence, and whether a pre-registered fallback was invoked.

### Success must be valid in the market where it is claimed

- **Language and register.** Language, honorifics, tone and script must fit the recipient, institution and channel used in that market.
- **Identity and formats.** Names, addresses, phone numbers, dates, time zones, currencies and units must use locally valid formats.
- **Payment and verification.** The flow must respect locally available payment rails, authentication, identity checks and user-consent steps.
- **Law and consumer terms.** Taxes, mandatory fees, cancellation, refund, disclosure and consumer-protection conditions are evaluated under the market's applicable rules.
- **Operational reality.** Inventory, delivery coverage, business hours, public holidays, service areas and real operating schedules must be valid at the evaluation time.
- **Channel and handoff.** The system must use locally normal channels and hand control back to the user or a human at the market-appropriate approval boundary.
- **Task-specific validity.** Reservations preserve time and cancellation constraints; transit uses real service and accessibility data; civic and healthcare administration use the correct jurisdiction, forms and non-clinical process.

### Commerce results report proximity to the lowest valid total

- **Equivalent offer.** Product identity, specification, quantity, destination, delivery deadline, service level and cancellation conditions must be equivalent before prices are compared.
- **All-in total.** The comparison uses the estimated payable total, including item price, delivery, taxes, service charges and payment fees.
- **Eligible discounts.** Coupons, memberships and personalized offers count only when the test persona is eligible; the requirement and redemption state are recorded.
- **Valid baseline.** The reference is the lowest constraint-satisfying total observed across the pre-registered representative platforms, with timestamp, availability and evidence lineage.
- **Price proximity.** MICA reports the chosen total, reference total, absolute difference and percentage premium. No result is called the market-wide absolute lowest price.
- **Constraints outrank price.** A cheaper option is excluded when it violates quality, safety, seller reliability, delivery, cancellation or another declared user constraint.

### The integration coverage contract

- **Access parity.** Surface and authorisation are fixed conditions held equal across systems. They describe what a journey requires here, never how hard the task is, and they carry no bonus and no penalty.
- **Handoff correctness.** Where the task contract says so, stopping at a one-time code, an identity approval, a QR payment or a card approval is a correct completion. The run must preserve enough state for the user to continue.
- **Completion is not a tool response.** When completion is asynchronous or happens off the agent's surface, a successful call is not proof of the final state. A receipt, an authoritative response or a state readback is required.
- **Retry and idempotency.** Retrying an action that is not idempotent must be guarded and recorded, so a duplicate order, booking or payment can be detected rather than inferred.
- **Evidence linkage.** A validated localized attempt names the market situation it exercised. The link is checked against the market's own declared situations, so a situation from another market or an unknown one is rejected.
- **What this is not.** It is a coverage plan, not a result. No task claims to cover every situation, no system has attempted one, and MICA names no vendor endpoint, private interface or third-party server anywhere in it.
- **Market integration.** Six markets carry a declared profile. Nothing in it has been exercised, and no market is described as covered by evidence.

### What this method cannot tell you

- **1.** Simulator results are reproducible but synthetic; real services change under you in ways a replica does not.
- **2.** Live-shadow runs stop at the confirmation boundary, so the final irreversible step is inferred rather than observed.
- **3.** Cells are small. A difference inside the 95% interval is not a difference.
- **4.** Cost depends on the operator's pricing on the snapshot date and moves independently of the system's behaviour.
- **5.** Coverage is uneven across markets, and an uncovered market is reported as missing rather than estimated.
- **6.** Speed and cost references are market-specific, so country scores compare distance to a local reference rather than absolute performance across countries.
- **7.** The derived task score has no uncertainty interval yet and is not used for a headline ranking during the pilot.
- **8.** None of the ten evaluation families carries a published result yet, so nothing on this site describes how any system behaves.

## 현재 운영 계약

### 공개 규칙

- **최소 유효 실행 수.** 미정. MICA는 최소 표본 크기를 합의하지 않았으므로 어떤 셀도 그 기준으로 판단할 수 없습니다. 관문은 숫자를 짐작하는 대신 거부합니다.
- **최소 커버리지.** 미정. 한 시장의 표준 과제 중 몇 퍼센트를 시도해야 하는지 MICA가 합의하기 전까지, 커버리지는 통과 조건이 될 수 없습니다.
- **임계값.** 이 에디션에는 공개 기준값이 설정되어 있지 않습니다. MICA가 과제별 실행 횟수, 최소 셀 표본 크기, 과제 커버리지, 최대 부적격 제외율을 아직 합의하지 않았으므로, 어떤 결과도 공개 대상으로 표시될 수 없습니다.
- **중대 안전 사건.** 중대 안전 사건이 하나라도 있으면 해당 셀의 공개를 영구히 차단합니다. 차단된 셀을 상위 집계에서 제외하지 않고 그 위의 계열·국가 수치 자체를 비공표합니다. 재실행으로 사건을 지우지 않습니다.
- **검증.** 독립 재실행만 인정합니다. 잠정 결과와 자체 보고 결과는 투명성을 위해 표시되며 공식으로 공개되지 않습니다.
- **데이터 상태.** 데모와 프리뷰 데이터는 어떤 경우에도 공개 대상이 될 수 없습니다. 이는 스키마 계층에서 강제되며, 데모 레코드가 그렇지 않다고 주장하면 빌드 시점에 오류를 냅니다.

### 세 개의 결과 축

- **정확도 · 확인된 최종 상태에 도달한 유효 실행의 비율.** 시스템이 확인 경계 안에서, 복구 불가능한 오류 없이, 과제가 선언한 최종 상태에 도달했는가?
- **속도 · 실제 경과 초, 성공한 유효 실행만.** 성공한 실행이 처음부터 끝까지 걸린 시간. 빠른 실패가 빠른 성공으로 읽히지 않도록 실패한 실행은 제외합니다.
- **비용 · 성공한 유효 실행당 통화 금액.** 전체 유효 시도 비용을 성공한 유효 실행 수로 나눈 값. 실패의 비용은 그 실패를 거쳐 도달한 성공에 부과됩니다.

### 원값 세 축을 먼저, 파생 감사값 하나를 다음에

- **과제별 최종 점수.** 100 × 정확도 × 속도 × 비용 각 인자는 원래의 초나 달러가 아니라 0과 1 사이로 정규화된 성분이므로, 곱한 값은 0에서 100 사이에 놓입니다.
- **정확도 성분.** 확인된 성공이면 1, 그 밖의 모든 결과는 0입니다. 부분 점수가 없으므로 더 빠르거나 더 싼 실패도 영점입니다. 곱셈이 그 영을 지나가기 때문입니다.
- **속도 성분.** min(1, 속도 기준값 ÷ 관측 초)입니다. 기준값은 실행 가능한 검증 과제 버전마다 사전에 등록하며 채점 대상 집단에서 다시 도출하지 않습니다. 그래서 느린 집단이 느린 시스템을 빨라 보이게 만들 수 없습니다. 기준값을 앞질러도 1에서 멈춥니다.
- **비용 성분.** min(1, 비용 기준값 ÷ 계측된 평가 실행 비용(USD))입니다. 비용이 영이거나 계측되지 않았으면 만점을 주지 않고 미측정으로 기록합니다.
- **계열 점수와 국가 점수.** 해당 계열 또는 시장의 적격 실행에서 파생한 과제 점수의 산술평균입니다. 사전 등록한 canonical 과제 집합이 완결되지 않으면 비공표합니다. 제외 항목은 분모에서 조용히 사라지지 않고 사유와 함께 남깁니다.
- **시장과 항목을 가로지르는 종합.** 정의되어 있지 않습니다. 단일 종합 수치를 내려면 시장과 항목에 걸친 가중치가 필요한데 MICA는 그것을 합의하지 않았습니다. 그래서 공개하지 않으며, 리더보드 구조가 그런 수치를 암시하지도 않습니다.
- **점수가 없는 과제.** 시도되지 않았거나 유효하지 않거나 사전 등록된 기준값이 없는 과제에는 점수 자체가 없습니다. 사유와 함께 평균에서 제외하며 0으로 세지 않습니다.
- **원값 공개.** 성공 여부 원값, 실제 경과 시간 원값, 평가 실행 비용 원값은 모든 점수 옆에 고유 단위 그대로 공개되며, 점수와 무관하게 검증할 수 있습니다.
- **이 식에서 비용의 뜻.** 평가 실행 비용, 즉 과제를 실행하는 데 든 모델·도구·API 비용이며 단위는 USD입니다. 에이전트가 처리한 상품 가격, 예약 금액, 거래 금액이 아니므로 어떤 환산도 곱셈에 끼어들지 않습니다.
- **현재 상태.** 측정된 시스템이 없고 점수를 가진 과제도 없습니다. 이 식은 숫자를 만들어 내기 전에 반박받도록 초안으로 공개합니다.

### 유효성과 분모

- **유효 실행.** 이란 심사를 통과한 시도를 말합니다. 환경에 접근할 수 있었고, 페르소나와 그 계정이 선언된 시작 상태에 있었으며, MICA 측 결함이 실행을 방해하지 않은 경우입니다. 유효하지 않은 시도는 실패로 세지 않고 채점 전에 버립니다.
- **정확도.** 는 성공한 유효 실행을 유효 실행으로 나눈 값이며, 95% 윌슨 점수 신뢰구간과 함께 보고합니다. 정규근사 대신 윌슨 구간을 쓰는 이유는 MICA의 셀이 작고 값이 0이나 1 근처에 놓이는 일이 잦은데, 그 구간에서 정규근사가 잘못 작동하기 때문입니다.
- **속도.** 는 성공한 유효 실행의 실제 경과 시간만 사용하며 p50과 p95로 보고합니다. 실패한 실행의 시간도 측정해 실행 셀에 보관하지만, 공개 백분위수에서는 의도적으로 제외합니다. 포함하면 빨리 포기하는 시스템이 빠른 것으로 보일 수 있기 때문입니다. 따라서 속도 수치는 성공한 실행만을 대상으로 하며, 정확도의 분모인 전체 유효 실행보다 대상 수가 적습니다.
- **비용.** 은 모든 유효 시도의 총비용을 성공 횟수로 나눈 값이며, 해당 시장의 통화로 표시합니다. 실패의 비용은 그 실패를 거쳐 도달한 성공에 부과됩니다. 성공이 0이면 값 자체가 존재하지 않으므로, 표는 0이나 무한대를 보여 주는 대신 “성공한 과제 없음.”라고 적습니다.
- **국가 매크로 평균.** 여러 시장에 걸친 수치에는을 사용합니다. 국가별 값의 평균이며, 모든 시장이 갖춰졌을 때만 계산합니다. 한 시장이 빠지면 그것을 0으로 다루는 대신 전체 수치를 보류합니다.

### 하나의 시스템, 여러 모델

- **과제별 라우팅.** 시스템은 과제마다 다른 모델로 보낼 수 있고, 그 과제에 가장 적합하다고 판단한 모델을 고를 수 있습니다.
- **복수 호출.** 한 번의 과제 시도가 여러 모델을 호출할 수 있습니다. 이는 정당한 설계 선택이며 그 자체로 감점되지 않습니다.
- **호출 계보.** 모든 호출은 근거로 공개됩니다. 공급자, 모델, 버전, 목적, 토큰, 비용, 지연, 그리고 시도 안에서의 순서를 남깁니다.
- **무엇이 채점되는가.** 경로가 아니라 시도가 채점됩니다. 라우팅은 시스템 전체가 실제로 낸 정확도·속도·평가 비용을 통해서만 점수에 반영됩니다.

### 일곱 개의 진단 축

- **오케스트레이션.** 분해, 순서 배치, 재시도, 그리고 멈추기로 하는 판단. 같은 기반 모델을 쓰는 시스템들 사이의 격차를 만드는 가장 큰 관측 원인입니다.
- **모델 라우팅.** 각 하위 단계를 적절한 모델로 보내는지, 그리고 경로를 쓸 수 없을 때 무너지지 않고 완만하게 성능을 낮추는지.
- **메모리.** 페르소나 사실, 앞선 제약, 과제 진행 상태를 단계 사이에 유지하면서 사용자에게 다시 묻거나 스스로 모순되지 않는 능력.
- **도구·API 사용.** 시스템이 선언한 도구를 정확하고 최소한으로, 형식에 맞게 사용하는지. 부분 응답과 오류 응답의 처리도 포함합니다.
- **현지화.** 언어, 주소와 이름 형식, 현지 결제·신원 인프라, 공휴일, 그리고 시장별 서비스 관행.
- **안전.** 확인 경계를 지키고, 동의 없이 되돌릴 수 없는 행위를 피하며, 범위를 벗어난 자격 증명 사용을 거부하는지.
- **복구.** 단계가 실패한 뒤의 행동. 탐지, 진단, 대체 경로, 또는 상태를 깨끗이 남긴 채 정직하게 멈추기.

### 검증 상태와 결과 트랙

- **독립 재실행 · IND.** MICA가 제출된 시스템 스냅샷을 MICA 통제 계정과 인프라에서 직접 다시 실행했고, 전체 근거 추적을 보유합니다. 공개 트랙에 있습니다.
- **잠정 · PROV.** MICA가 부분 실행이나 감독하의 실행을 관찰했거나, 주장된 커버리지 중 일부에 대해서만 근거를 보유합니다. 결과는 알리되 공식 결과로 공개하지 않습니다. 공식으로 공개되지 않습니다.
- **자체 보고 · SELF.** 시스템 운영 주체가 제출했으며 MICA가 재현하지 않은 근거 추적입니다. 투명성을 위해서만 표시합니다. 공식으로 공개되지 않습니다.
- **시뮬레이터.** 흔한 현지 서비스 흐름을 결정론적으로 재현한 로컬 복제본. 재현 가능하고 저렴하며, MICA가 시스템 간 동일 조건을 보장할 수 있는 유일한 트랙입니다.
- **라이브 섀도.** 시스템이 실제 서비스를 상대로 동작하되 확인 경계에서 멈춥니다. 되돌릴 수 없는 행위는 하지 않습니다. 시뮬레이터에서의 행동이 실제로도 이어지는지 확인하는 데 씁니다.
- **제한적 검증 라이브.** 서비스 운영 주체에게 사전 통지한 뒤, MICA가 보유한 테스트 계정에서 소수의 종단 간 실행을 수행합니다. 의도적으로 범위가 좁습니다.

### 각 문구의 뜻

- **성공한 과제 없음..** 유효한 시도는 있었으나 어느 것도 선언된 최종 상태에 도달하지 못했으므로, 속도와 성공당 비용은 정의되지 않습니다.
- **이 시장에 커버리지 없음..** 이 시장 또는 이 조건에 대해 시스템에 유효 실행 셀이 아예 없습니다. 나쁜 결과가 아니라 근거의 부재입니다.
- **측정하지 않음.** 이 스냅샷에서 해당 축은 평가하지 않았습니다. 낮은 값이 아닙니다.

### 측정, 해석, 권고

- **측정.** 실행 기록에서 직접 읽은 값. 추론 없음.
- **MICA 해석.** 측정값이 무엇을 뜻하는지에 대한 MICA의 읽기. 반박 가능하며, 에디션의 이름으로 서명됩니다.
- **권고.** 만드는 사람과 사는 사람을 위한 조언. 결코 측정의 주장이 아닙니다.

### 국가별 대표 플랫폼은 실행 전에 고정합니다

- **대표 후보군.** MICA는 시장과 과제마다 그 시장의 일반 소비자가 실제로 널리 이용하는 플랫폼 후보군을 현지 도달 범위, 현재 가용성, 과제 관련성 근거와 함께 사전 등록합니다.
- **기능 적합성.** 선정 플랫폼은 과제의 선언된 최종 상태, 제약 조건, 확인 경계를 바꾸지 않고 표준 과제를 지원해야 합니다.
- **접근 동등성.** 모든 평가 시스템에 같은 선정 플랫폼, 계정 등급, 시작 상태, 권한, 확인 경계를 제공합니다.
- **사전 등록과 대체.** 선정 플랫폼, 버전, 스냅샷 날짜, 대체 순서를 실행 전에 고정합니다. 대체 플랫폼은 문서화된 장애 때만 사용하며 특정 시스템에 더 쉽다는 이유로 바꾸지 않습니다.
- **근거 계보.** 공개 실행 셀에는 실제 사용 플랫폼, 버전과 스냅샷 날짜, 선정 근거, 사전 등록된 대체 플랫폼 사용 여부를 기록합니다.

### 성공은 그 결과를 주장하는 시장에서 유효해야 합니다

- **언어와 격식.** 언어, 존대, 어조, 문자는 해당 시장의 수신자, 기관, 채널에 맞아야 합니다.
- **신원과 형식.** 이름, 주소, 전화번호, 날짜, 시간대, 통화, 단위는 현지에서 유효한 형식을 사용해야 합니다.
- **결제와 인증.** 현지에서 이용 가능한 결제 수단, 인증, 신원확인, 사용자 동의 절차를 따라야 합니다.
- **법규와 소비자 조건.** 세금, 의무 수수료, 취소, 환불, 고지, 소비자보호 조건은 해당 시장의 적용 규칙으로 평가합니다.
- **운영 현실.** 재고, 배송권역, 영업시간, 공휴일, 서비스 지역, 실제 운행 일정은 평가 시점에 유효해야 합니다.
- **채널과 인계.** 현지에서 통상 사용하는 채널을 쓰고, 시장에 맞는 승인 경계에서 사용자나 사람에게 제어권을 돌려줘야 합니다.
- **과제별 유효성.** 예약은 시간과 취소 조건을 지키고, 이동은 실제 운행과 접근성 데이터를 사용하며, 행정과 의료 행정은 올바른 관할, 서식, 비임상 절차를 따라야 합니다.

### 커머스 결과는 최저 유효 총액과의 근접도를 따로 보고합니다

- **동등한 제안.** 상품 식별자, 규격, 수량, 배송지, 도착 기한, 서비스 등급, 취소 조건이 동등한 경우에만 가격을 비교합니다.
- **최종 예상 총액.** 상품 가격, 배송비, 세금, 서비스비, 결제 수수료를 포함한 최종 결제 예상 총액을 비교합니다.
- **사용 가능한 할인.** 쿠폰, 회원등급, 개인화 혜택은 테스트 페르소나가 실제로 사용할 수 있을 때만 반영하고 요건과 적용 상태를 기록합니다.
- **유효한 기준값.** 기준값은 사전 등록된 대표 플랫폼에서 제약을 만족하며 확인된 최저 총액이고, 시각, 가용성, 근거 계보를 함께 남깁니다.
- **가격 근접도.** 선택 총액, 기준 총액, 절대 차이, 비율 프리미엄을 보고합니다. 이를 시장 전체의 절대 최저가라고 부르지 않습니다.
- **제약 조건 우선.** 더 싼 선택지가 품질, 안전, 판매자 신뢰도, 배송, 취소 또는 사용자의 다른 선언 제약을 위반하면 후보에서 제외합니다.

### 연동 커버리지 계약

- **접근 조건의 동일 적용.** 접점과 승인 경계는 모든 시스템에 동일하게 적용되는 고정 조건입니다. 이곳의 과제가 무엇을 요구하는지를 나타낼 뿐 난이도를 뜻하지 않으며 가점도 감점도 없습니다.
- **올바른 인계.** 과제 계약이 그렇게 정한 경우, 일회용 코드나 신원 승인, QR 결제, 카드 승인 지점에서 멈추는 것은 올바른 완료입니다. 사용자가 이어서 진행할 수 있도록 상태를 보존해야 합니다.
- **호출 성공은 완료가 아닙니다.** 완료가 비동기이거나 에이전트 화면 밖에서 이루어질 때, 호출이 성공했다는 사실은 최종 상태의 증거가 아닙니다. 영수증이나 권위 있는 응답, 또는 상태 재확인이 필요합니다.
- **재시도와 멱등성.** 멱등하지 않은 동작을 다시 시도할 때는 반드시 보호하고 기록해야 합니다. 그래야 중복 주문, 중복 예약, 중복 결제를 추정이 아니라 확인으로 잡아낼 수 있습니다.
- **근거 연결.** 검증된 현지 실행 기록은 자신이 수행한 시장 상황을 명시합니다. 이 연결은 해당 시장이 선언한 상황 목록과 대조하므로, 다른 시장의 상황이나 존재하지 않는 상황은 거부됩니다.
- **이것이 아닌 것.** 이는 커버리지 계획이며 결과가 아닙니다. 모든 상황을 포괄한다고 주장하는 과제는 없고, 어떤 시스템도 시도한 적이 없으며, 특정 사업자의 접속 지점이나 비공개 인터페이스, 외부 서버를 지목하지도 않습니다.
- **시장 연동.** 6개 시장이 선언된 프로필을 갖고 있습니다. 그중 수행된 것은 없으며, 근거로 커버되었다고 표현하는 시장도 없습니다.

### 이 방법이 알려 줄 수 없는 것

- **1.** 시뮬레이터 결과는 재현 가능하지만 합성입니다. 실제 서비스는 복제본이 흉내 내지 못하는 방식으로 사용자 아래에서 바뀝니다.
- **2.** 라이브 섀도 실행은 확인 경계에서 멈추므로, 마지막 되돌릴 수 없는 단계는 관측이 아니라 추론입니다.
- **3.** 셀이 작습니다. 95% 신뢰구간 안쪽의 차이는 차이가 아닙니다.
- **4.** 비용은 스냅샷 날짜의 운영 주체 가격 정책에 좌우되며, 시스템의 행동과 무관하게 움직입니다.
- **5.** 커버리지는 시장마다 고르지 않으며, 커버되지 않은 시장은 추정하지 않고 없음으로 보고합니다.
- **6.** 속도·비용 참조값은 시장마다 다르므로 국가 점수는 국가 간 절대 성능이 아니라 현지 기준과의 거리를 비교합니다.
- **7.** 파생 과제 점수에는 아직 불확실성 구간이 없으며, 파일럿 동안 대표 순위에 사용하지 않습니다.
- **8.** 10개 평가 계열 중 어디에도 아직 게시된 결과가 없으므로, 이 사이트의 어떤 내용도 시스템의 실제 동작을 설명하지 않습니다.

## 현재 정책 등록부 (Korean policy register)

MICA가 책임지겠다고 밝히는 현재의 진술입니다. 모든 줄이 현재형이며, 이 등록부는 이력이 아니고 이전 값을 기록하지 않습니다.

- **프로젝트 정의.** MICA, 곧 다국적 소비자 에이전트 지수는 버전이 고정된 완성 상태의 소비자 에이전트 시스템을 평가하는 벤치마크입니다. 표준 공개 도메인은 micabench.com입니다.
- **시장 범위.** 지수가 다루는 시장은 6개입니다. KR, JP, SG, TW, AE, TH이며, 이것이 지수의 범위이고 그 밖의 시장은 포함되지 않습니다.
- **과제 목록.** 10개 과제 계열에 100개의 표준 정의가 있습니다. 모두 공개 세트의 잠정 후보이며, 실행 가능한 검증 시나리오도, 측정된 결과도, 홀드아웃도 아닙니다.
- **실행별 파생 감사 점수.** 정확도·지연·계측된 평가 비용의 원값이 기록이며 각각 따로 공개됩니다. 유효하고 검증된 실행마다 100 × 정확도 × 속도 × 비용을 파생할 수 있지만, 파일럿 동안 이 곱은 대표 결과나 순위 기준이 아닌 감사값입니다. 사전 등록한 canonical 과제 집합이 완결되지 않으면 계열·시장 수치는 비공표합니다. 이 곱셈은 MICA의 정책이며 외부 문헌에서 도출한 결과가 아닙니다.
- **공개 후보와 홀드아웃.** 공개 후보 세트와 비공개 홀드아웃 세트는 분리되어 있으며, 분리는 내보내기 단계에서 강제됩니다. 항목마다 생성, 노출, 버전, 출처를 기록합니다.
- **시장 연동 프로필.** 6개 시장 에디션은 각각 대표 표면, 승인 경계, 완료 의미, 복구 조건, 근거 요건을 선언합니다. 이는 계획된 커버리지를 선언할 뿐 측정된 커버리지를 뜻하지 않습니다.
- **중립 거버넌스 의도.** MICA는 중립이어야 합니다. vooy는 발의자이자 도전 시스템일 수 있으나, 규범적 공개 설계에서 벤치마크 소유자로서의 특권적 지위를 갖지 않습니다. 운영 주체, 결정 권한, 자금, 유상 과제 저작 관계, 공급자 관계, 이해충돌을 공개합니다. 구조적 독립성은 해결된 주장이 아니라 미해결 요건입니다.