Evidence

Evidence for clinical AI translation and hardening.

Current evidence supports checked synthetic clinical worlds and bounded diagnostic workflows. It does not yet establish a completed closed-loop hardening result or broad real-world model improvement.

Current strongest result

96 checked sequential hourly transitions.

Synset has internally validated acute-care trajectory generation across four days of hourly clinical evolution. This supports generated-artifact checking, not a completed hardening claim.

96

Sequential hourly transitions

Validated acute-care trajectory horizon.

97

State points

Initial anchor plus 96 transitions.

14,143

Accepted acute candidates

Accepted-only validator audit population.

0

Accepted audit violations

Observed among accepted acute candidates.

Caveat: Accepted-only validator audit, not held-out future patient accuracy, clinical truth, regulatory approval, or synthetic-control-arm validity. It does not show that targeted synthetic data improved a model.

Methodology note

The reported population contains 14,143 accepted acute candidates from Synset's 96-hour trajectory workflow. The audit is accepted-only: it reports observed violations among material that passed the release process, not performance across all generated candidates or future patient outcomes.

Release certificates

A certificate is more than a sealed report.

A hash can show that a file has not changed. That is useful, but it is not clinical validation. An integrity seal tells you the evidence was not altered. A Synset certificate records why the material passed the checks required for its stated use.

Release record

Release Certificate

  • Clinical fitPASS
  • Population supportPASS
  • Timeline consistencyPASS
  • Metadata safetyPASS
  • Text groundingPASS

Clinical fit

Values and transitions satisfy stated clinical constraints.

Population support

Patterns remain consistent with empirical cohort-level structure.

Timeline consistency

Patient state changes remain plausible across time.

Metadata safety

No source IDs, internal linkage, or forbidden identifiers.

Text grounding

Notes, dialogue, and claims-style outputs stay tied to structured evidence.

Evidence tiers

Artifact checks, model diagnosis, and hardening proof are separate.

Certificates govern generated material. Held-out real data determines whether an intervention met its prespecified evidence standard.

Tier 1

Generated-world release evidence

Records whether synthetic clinical worlds passed the clinical, timing, population-support, metadata, and grounding checks for their stated use.

Supported now for bounded generated artifacts. A release certificate does not establish downstream model improvement.

Tier 2

Bounded diagnostic evidence

Maps failure surfaces with declared scenarios, model-facing evidence, rubrics, response scoring, and explicit use limits.

The diagnostic workflow is implemented for bounded assessments. A public outcome case has not yet been published.

Tier 3

Closed-loop hardening evidence

Compares frozen original and updated models on held-out real data using prespecified success and no-regression criteria.

Not yet complete as a public proof asset. Broad model-improvement claims remain out of scope until this evidence and external replication exist.

Technical evidence package

What the record behind an engagement contains.

The record keeps the model use, failure hypothesis, generated material, intervention, evaluation protocol, results, and claim boundary distinct.

Private model artifacts, patient-level evaluation data, raw responses, reviewer notes, internal run logs, hidden rubrics, and source-derived details are not public by default.

  • declared model use and evaluation scope
  • failure-surface and correctability summary
  • synthetic-world release-check summary
  • original and updated model references
  • frozen intervention and evaluation versions
  • held-out evaluation reference, if applicable
  • success and no-regression summary
  • public/private evidence classification
  • residual-risk and claim-boundary statement
Failure discovery

Find the blind spot before prescribing the fix.

Checked clinical worlds can expose gaps that ordinary retrospective validation misses. A useful result may be an improved candidate, a requirement for new real labels, a narrower intended use, or a defensible decision not to proceed.

  • missingness
  • delayed signal
  • care-pathway variation
  • subgroup presentation
  • documentation drift
  • longitudinal drift
  • counterfactual inconsistency
  • unsafe reassurance
  • failure to ask for missing context
  • contraindication misses
Hardening evidence standard

Held-out real data tests whether the intervention worked.

Held-out real data is a customer-controlled real evaluation set that is not used to select, tune, or choose the hardening intervention. It determines whether the updated model met the prespecified success and no-regression criteria within the declared evaluation scope.

  • holdout designated before intervention selection
  • prespecified success criteria
  • prespecified no-regression checks
  • frozen intervention before final evaluation
  • original and updated model versions
  • calibration and subgroup review
  • no iterative optimization against final holdout results
  • residual-risk and stop-signal report
What Synset is proving next

The next proof is a closed loop.

Synset must identify one clinically meaningful weakness, generate targeted hardening material, train or support an updated model, and demonstrate a prespecified improvement on held-out real data without unacceptable regression.

Next gate

one supported clinical model and declared use

Next gate

frozen original and updated model versions

Next gate

customer-controlled held-out real evaluation data

Next gate

prespecified success and no-regression criteria

Next gate

external replication and a public-safe report

Research and roadmap directions

These are roadmap or partner-driven directions, not current validated claims unless a specific evidence artifact says otherwise.

  • broader autonomous remediation across model classes
  • patient-conditioned assurance, not patient-specific prediction
  • life-sciences and real-world-evidence exploration
  • imaging, genetic, waveform, and voice modalities
  • trial-feasibility exploration, not synthetic control arms

Start with a readiness assessment.

Start with one supported model, one declared use, one suspected weakness, and whether a held-out evaluation set exists and where it can be evaluated. Do not send patient-level data by email.