Question Types & Input
Send an evaluation item (input) plus one or more rubric questions. Answers come back in the same order, one per question, typed to match.
A complete request
{
"input": [
{
"role": "user",
"content": [
{
"type": "input_text",
"text": "Fictional offline AI evaluation. Source: Medication allergies are not documented. Model summary: The patient has no known medication allergies. Rubric: Preserve uncertainty; missing documentation is not a confirmed negative finding. Evaluate the summary, not patient care."
},
{
"type": "input_json",
"data": {
"dataset": "summary-evaluation",
"version": "v2"
}
}
]
}
],
"questions": [
{
"type": "predicate",
"name": "unsupported_claim",
"instructions": "Does the summary make a clinical assertion unsupported by the source?"
},
{
"type": "choice",
"name": "error_category",
"instructions": "Classify the summary against the source-fidelity rubric.",
"choices": [
{
"value": "none",
"description": "Preserves the source meaning"
},
{
"value": "unsupported_certainty",
"description": "Turns missing information into a confirmed finding"
},
{
"value": "insufficient_context",
"description": "Cannot assess from the supplied material"
}
]
},
{
"type": "score",
"name": "error_severity",
"instructions": "Rate the summary error using these ordered levels.",
"levels": [
{
"label": "None",
"description": "No error"
},
{
"label": "Minor",
"description": "Wording issue without a change in clinical meaning"
},
{
"label": "Material",
"description": "Changes a clinical fact or its certainty"
}
]
}
],
"external_id": "eval_0042",
"include_commentary": true
}| Parameter | Type | Required | Description |
|---|---|---|---|
input | string | message[] | Required | The model output, context, and rubric. A nonempty string, or a nonempty array of user messages (see below). |
questions | question[] | Required | 1 to 50 questions, each with a unique name and up to 500 characters of question text. Three are included in the price; each additional question adds $1.00. |
reviewer | object | Optional | Who may answer. Defaults to any eligible physician. See Choosing a Reviewer. |
external_id | string | Optional | Your correlation ID, up to 256 characters, no control characters. Screened like everything else; never put identifiers in it. |
include_commentary | boolean | Optional | Default false. true requires the physician to add a brief comment (up to 280 characters) on every question, included in the price. |
escalate | boolean | Optional | Default false. true escalates in the same request (needs max_price_cents and an Idempotency-Key header). |
max_price_cents | integer | Optional | Required iff escalate is true. |
Unknown fields are rejected with 400 invalid_request and the offending path in error.field (for example questions[1].choices[0].weight), so content can never be silently dropped.
Input
The simplest input is a string. It is treated as one user message with one text part.
{
"input": "Fictional offline AI evaluation. Source: Medication allergies are not documented. Model summary: The patient has no known medication allergies. Rubric: Preserve uncertainty; missing documentation is not a confirmed negative finding. Evaluate the summary, not patient care.",
"questions": [
{
"type": "predicate",
"name": "unsupported_claim",
"instructions": "Does the summary make a clinical assertion unsupported by the source?"
}
]
}For structured cases, send an array of messages. Each message has role: "user" (or omit role), optionally type: "message", and content: a string or a nonempty array of parts.
| Part | Fields | Notes |
|---|---|---|
input_text | text | Plain text. |
input_json | data | An object, or an array of objects. Nested values can be any JSON. Up to 32 levels deep, 10,000 values, and 256 KiB in canonical compact form. Counted for billing as canonical JSON (see Pricing). |
input_image | image_url, optional detail | A base64 data URL (data:image/png;base64,…) or an https:// URL. detail is low, high, auto or original. |
Question instructions accept the same thing as message content: a string or an array of these three part types. Every content location follows the same screening, counting and image rules.
Images
- At most 3 images per request, counted across input and question instructions together.
- JPEG, PNG, HEIC, GIF or WebP. Up to 10 MB each and 40 megapixels.
http://URLs andfile_idreferences are rejected. Usehttps://or a base64 data URL.- An HTTPS image is fetched once, when you submit. The physician sees exactly the image that was screened and previewed; changing the file at that URL later has no effect. URLs that resolve to private or internal addresses are refused.
- Every image is normalized before anything else sees it: converted to a standard format, downscaled to at most 2560 px on the longest side, and stripped of all metadata, including location.
- Each image adds a flat 500 billable tokens. Images do not count toward the 10,000-token text limit.
- Images are screened for identifying content too: faces and visible text such as names, dates of birth or record numbers. Crop them out.
An image that can't be fetched or decoded returns 400 invalid_image naming the field.
Questions
Every question has a type, a name, and instructions. Names must be nonempty, unique within the request, at most 128 characters, and may not start with _appendix (reserved). Answers are matched to questions by name and returned in question order.
predicate
A yes/no question. No extra fields.
choice
Pick one of 2–255 choices. Each choice has a value and an optional description (up to 4,096 characters). Values are a nonempty string (up to 1,024 characters) or a boolean, and must be unique. Types are preserved: true and "true" are different values, and answers return the value with its original type.
{ "type": "choice", "name": "unsupported_claim",
"instructions": "Does the model summary add a fact not supported by the source?",
"choices": [ { "value": true }, { "value": false }, { "value": "insufficient_context" } ] }score
Pick one of 2–10 ordered levels. Each level has a nonempty label (up to 1,024 characters) and an optional description. Levels are identified by their 0-based position.
AI answers
ai.answers has one entry per question, in question order.
{
"ai": {
"answers": [
{
"type": "predicate",
"name": "unsupported_claim",
"probability": 0.95
},
{
"type": "choice",
"name": "error_category",
"choice": "unsupported_certainty",
"confidence": 0.9,
"probabilities": [
{
"value": "none",
"probability": 0.03
},
{
"value": "unsupported_certainty",
"probability": 0.94
},
{
"value": "insufficient_context",
"probability": 0.03
}
]
},
{
"type": "score",
"name": "error_severity",
"score": 1.9,
"confidence": 0.9,
"probabilities": [
{
"value": 0,
"label": "None",
"probability": 0.02
},
{
"value": 1,
"label": "Minor",
"probability": 0.06
},
{
"value": 2,
"label": "Material",
"probability": 0.92
}
]
}
]
}
}| Type | Fields |
|---|---|
predicate | probability: the model's probability, 0–1, that the answer is true. |
choice | choice: the top value (original type); confidence; probabilities: {value, probability} per choice. |
score | score: the overall answer on the level scale, from 0 (lowest level) to the highest level index; confidence; probabilities: {value, label, probability} per level, where value is the 0-based level index. |
refusal | The model declined to answer this question. name identifies the question. Other answers are unaffected. |
AI values are model outputs and are returned without rounding. Treat any additional fields on an AI answer as optional, and handle refusal wherever you read an answer. The physician's answers use a different shape; see Physician Answers.
Differences from the structured decision format
The request follows a widely used structured decision format for typed questions. It is not full compatibility: the Decisions API accepts a documented subset, and is stricter in a few places. If you already produce requests in that format, check them against this list:
- No
modelfield. Appendix selects the model. Sendingmodelis a 400. - User messages only. System, developer and assistant roles are rejected, not ignored.
- No tool items. Tool calls, tool outputs and other non-message items are rejected.
- Three part types.
input_text,input_jsonandinput_image. File references (file_id) andhttp://image URLs are rejected. - Names are required and unique. Every question needs a nonempty
name, so answers map back to questions without relying on position. - Up to 500 characters of text per question (its
instructions; put long context ininput), at most 3 images per request, at most 10,000 text and JSON tokens (and 100,000 characters), and at most 50 questions. - Appendix-only fields:
reviewer,external_id,include_commentary,escalateandmax_price_cents.
status, knowledge.status, cancellation_reason and specialty values as possible. New values may be added without notice; new request fields are optional additions.