Sealed candidate-blind synthetic proof
Supports a bounded synthetic-world behavioral claim under a frozen generation and evaluation policy.
Clinical AI hardening
Model Hardening, grounded in evidence.
Investigate consequential gaps, attempt an allowed intervention where justified, and use proof matched to the claim. Deep Auditing provides a diagnostic entry when the question is still unclear.
First deliverable · Evidence Profile
Start with what your evidence supports.
The customer brings an approved model interface or response export, its intended clinical use, the decision question, and the evidence available. The Evidence Profile maps support, common-case behavior, subgroup intersections, labels, calibration, missingness, chronology, confounds, and input fidelity. It identifies which questions can be answered, where support is weak or uncertain, and what deserves deeper investigation. You do not need to arrive with a known failure or a second dataset.
The Evidence Record is designed to connect the question, evidence, challenge, adjudication, any justified intervention, proof decision, limitations, and next decision. Its persistent platform layer is still being built.
See a fictional Evidence Record exampleModel Hardening
Attempt one bounded intervention
Investigate and adjudicate a consequential weakness, attempt a permitted intervention where justified, then use proof matched to the claim and preserve the complete result.
01 / Lead offering
Model Hardening
Buyer
Teams preparing for external validation, an enterprise pilot, a material model update, or a new population or site. Start with the model, intended use, and available evidence; a known failure or intervention is not an entry requirement.
What Synset packages
Investigate a consequential weakness, attempt an allowed intervention where justified, and evaluate the result under the declared proof mode.
Process
- 01Profile available evidence
- 02Run a controlled challenge
- 03Adjudicate the finding
- 04Apply a targeted intervention if justified
- 05Use claim-appropriate proof
- 06Preserve the Evidence Record and Regression Suite
Differentiation
Allowed changes may include preprocessing or interface repair, calibration, thresholds, prompts, retrieval or tool policy, routing, abstention, targeted data, and weight adaptation when permitted and compatible. API-only systems may allow some of these without weight access.
No improvement is guaranteed. A justified result may be no effect, remaining uncertainty, a narrower intended use, a need for more evidence, or a stop when no permitted intervention meets the criteria in scope.
correctability decision
targeted challenge or hardening material where justified
frozen intervention record
claim-appropriate proof plan
remaining-risk decision
Deep Auditing
Understand what the evidence supports
Begin with a systematic Evidence Profile, then investigate behavior where the profile shows weak support, uncertainty, or a consequential unanswered question.
02 / Diagnostic entry
Deep Auditing
Buyer
Teams that need to understand what their intended use and available evidence support, including when the consequential weakness is not yet known.
What Synset packages
Systematically assess evidence support and investigate model behavior. Return findings, uncertainty, and next steps without implying an intervention or improvement.
Process
- 01Define the intended use
- 02Profile available evidence
- 03Investigate behavior
- 04Adjudicate findings and uncertainty
- 05Recommend next steps
Differentiation
Deep Auditing is a diagnostic entry: the Evidence Profile and investigated findings help decide whether an intervention is justified. It can conclude with findings and next steps without implying a model change.
Evidence Profile of available support and uncertainty
investigated behavior and adjudicated findings
limits on conclusions the evidence can support
recommended next decision
Proof follows the claim
Three evidence pathways, with different boundaries.
Patient- or episode-grouped customer cross-fitting
Can support a customer-scoped procedure claim from one labeled dataset when each evaluation group is excluded from the full fold-specific adaptive pipeline.
Untouched, source-distinct, temporal, or prospective evidence
Supports the corresponding stronger claims. The required evidence depends on the claim; no single real-world holdout is a universal entry condition.
Proof does not guarantee improvement. Frozen materiality criteria, no-regression checks, and independent clinical or evaluator authority remain appropriate to the declared claim.
Who this is for
Built for teams moving clinical AI into practice.
Clinical AI startups
Teams moving from a promising prototype toward an external pilot or evidence-backed product decision.
Translational laboratories and physician-led spinouts
Research teams that need a practical path from a measured model result to a bounded development plan.
Digital-health vendors
Product teams preparing models and workflows for enterprise evaluation and deployment.
Healthcare and life-sciences model teams
Builders working with sparse, delayed, or underrepresented clinical data and patient contexts.
Lifecycle deliverables and future direction
Preserve evidence as the system changes.
The Evidence Record and Regression Suite carry findings through later model and workflow changes. They are lifecycle deliverables, not additional launch products.
Longer-term direction
Clinical Hardening Library
Over time, and where contracts permit, Synset may develop reusable blind-spot methods, controlled challenge templates, and evidence about when interventions succeed or fail.
Customer-specific findings remain segregated. Reusable methodology is retained only where contractually permitted.
Building
Evidence Record
Persistent traceability across model versions, identified gaps, intervention protocols, evaluation results, evidence limitations, and future updates.
The current product produces structured reports and artifacts; the persistent platform layer is still being built.
Preview the Evidence Record →Today / recurring expansion
Regression Assets
Fixed customer-specific tests that preserve discovered blind spots, important evidence conditions, previously identified regressions, corrected regressions where correction has been demonstrated, and future model-update checks.
Customer-specific regression cases are not reused across customers without authorization.
Deep Auditing scope
Questions the diagnostic engagement can investigate
Documentation, coding, and risk-model behavior are examples of assessment scope within Deep Auditing, not separate product identities. The Evidence Profile determines which conclusions the available evidence can support.
Example audit scope
Documentation Model Assessment
A bounded assessment of unsupported assertions, omitted critical facts, contradictions, medication or lab mismatches, temporal errors, and sensitivity to documentation shifts.
Assess a documentation modelExample audit scope
Coding / CDI Model Assessment
A bounded assessment of missed codes, unsupported codes, upcoding risk, diagnosis hallucination, and inconsistency between notes and structured evidence.
Assess a coding modelExample audit scope
Risk Model Assessment
A bounded assessment of calibration, false negatives, subgroup behavior, missingness, drift, and risk coverage under controlled clinical conditions.
Assess a risk modelSupporting materials
Controlled material used within development and QA.
Scenario material, records, and QA fixtures may support a defined challenge or workflow. These are supporting materials within the two offerings, not additional waitlist products.
Supporting material
Synthetic Scenario and QA Packages
Checked patient scenarios, records, timelines, notes, and controlled variants for development, QA, demonstrations, benchmarks, and bounded assessments.
Request a scenario or QA packageSupporting material
FHIR / EHR Sandbox Blueprint
Clinically coherent, longitudinal, scenario-controlled test populations for integration tests, staging, demonstrations, and workflow simulation without PHI.
Request sandbox blueprintSupporting material
Rare-State Scenario Pack
Bounded scenario families for rare states, sparse evidence, unusual combinations, and clinically important edge conditions.
Discuss a rare-state packHow an engagement runs
Customer provided
1. Establish approved access
The customer provides an approved model interface or response export, declared clinical use, target or expected-output definition, and available evidence. The initial profile can establish what evaluation paths are feasible.
Synset-supported
2. Map the contract
Synset validates the input and output contract, records model and data versions, identifies candidate failure surfaces, and defines the evaluation scope with the customer.
Synset workflow and review
3. Generate and check scenarios
A controlled challenge is designed for the adjudicated question. When synthetic material is appropriate, its generation and evaluation policy are frozen before proof.
Agreed with the customer
4. Select the intervention
Permitted changes may include preprocessing or interface repair, calibration, thresholds, prompts, retrieval or tool policy, routing, abstention, targeted data, or weight adaptation when allowed and compatible. No intervention is recommended when none is justified.
Customer-controlled evaluation
5. Compare model versions
Use sealed candidate-blind synthetic proof for a bounded synthetic-world claim; patient- or episode-grouped customer cross-fitting for a customer-scoped procedure claim; or untouched, source-distinct, temporal, or prospective evidence for corresponding stronger claims. Keep each evaluation group out of its full adaptive pipeline.
Workflow output and review
6. Preserve the result
The engagement produces an evidence record, remaining-risk map, regression tests, and a recommendation to proceed, collect more real evidence, narrow the use, or stop.
Reusable software layer: versioned probe suites, scenario generation, artifact checks, manifests and hashes, response import, scoring and reporting, and regression cases.
Human or partner work: intended-use definition, clinical review, intervention selection, model-specific integration, regulatory interpretation, and release decisions.
Automation depends on model type and access. Current hardening pilots remain bounded, versioned engagements rather than universal autonomous retraining. Timing is scoped during intake.
Current model scope
- Scenario-based text systems can be assessed when responses can be imported or reached through an approved OpenAI-compatible endpoint. This can include clinical assistants, RAG workflows, documentation models, and coding/CDI models.
- Structured prediction models, including risk models, are scoped case by case and require reproducible inference, a defined target, and suitable evaluation data. Longitudinal or survival models require separate review of time horizon, target, censoring, and validation design.
- API-only and closed commercial systems may be assessed when responses can be captured through an approved path. Some allow relevant changes to the interface, prompt, retrieval or tool policy, routing, thresholds, calibration, or abstention without weight access. If no permitted relevant intervention is available, the system may be audited but not hardened in that engagement.
- A second dataset is not universally required. Proof follows the claim and available evidence; stronger real-world claims require the corresponding real-world evidence. Imaging, waveform, voice, and unsupported modalities remain partner-driven.
Roadmap and partner-driven directions
These are not current validated products unless a specific evidence artifact marks a use case as validated.
Imaging descriptors
Roadmap or partner-driven unless a specific evidence artifact marks a use case as validated.
Waveform descriptors
Roadmap or partner-driven support for workflow stress testing, not a validated current modality.
Clinical speech and dialogue simulation
Roadmap or partner-driven unless separately released with evidence and claim boundaries.
Life-sciences / RWE exploration
Exploration area for partner studies, not a public claim of broad real-world evidence validity.
Trial-feasibility exploration
Not synthetic control arms as a current product, and not a replacement for real-world validation.
Start with the Evidence Profile.
Model Hardening is the lead offering. Deep Auditing is the diagnostic entry when you need findings and next steps before deciding whether an intervention is justified.
Two ways to engage
Model Hardening and Deep Auditing.
The Evidence Profile, Evidence Record, and Regression Suite are deliverables across the lifecycle, not additional waitlist products. No result or pilot access is guaranteed.