Skip to main content

Question Types & Input

Send an evaluation item (input) plus one or more rubric questions. Answers come back in the same order, one per question, typed to match.

A complete request

POST /api/v1/decisions
{
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Fictional offline AI evaluation. Source: Medication allergies are not documented. Model summary: The patient has no known medication allergies. Rubric: Preserve uncertainty; missing documentation is not a confirmed negative finding. Evaluate the summary, not patient care."
        },
        {
          "type": "input_json",
          "data": {
            "dataset": "summary-evaluation",
            "version": "v2"
          }
        }
      ]
    }
  ],
  "questions": [
    {
      "type": "predicate",
      "name": "unsupported_claim",
      "instructions": "Does the summary make a clinical assertion unsupported by the source?"
    },
    {
      "type": "choice",
      "name": "error_category",
      "instructions": "Classify the summary against the source-fidelity rubric.",
      "choices": [
        {
          "value": "none",
          "description": "Preserves the source meaning"
        },
        {
          "value": "unsupported_certainty",
          "description": "Turns missing information into a confirmed finding"
        },
        {
          "value": "insufficient_context",
          "description": "Cannot assess from the supplied material"
        }
      ]
    },
    {
      "type": "score",
      "name": "error_severity",
      "instructions": "Rate the summary error using these ordered levels.",
      "levels": [
        {
          "label": "None",
          "description": "No error"
        },
        {
          "label": "Minor",
          "description": "Wording issue without a change in clinical meaning"
        },
        {
          "label": "Material",
          "description": "Changes a clinical fact or its certainty"
        }
      ]
    }
  ],
  "external_id": "eval_0042",
  "include_commentary": true
}
ParameterTypeRequiredDescription
inputstring | message[]RequiredThe model output, context, and rubric. A nonempty string, or a nonempty array of user messages (see below).
questionsquestion[]Required1 to 50 questions, each with a unique name and up to 500 characters of question text. Three are included in the price; each additional question adds $1.00.
reviewerobjectOptionalWho may answer. Defaults to any eligible physician. See Choosing a Reviewer.
external_idstringOptionalYour correlation ID, up to 256 characters, no control characters. Screened like everything else; never put identifiers in it.
include_commentarybooleanOptionalDefault false. true requires the physician to add a brief comment (up to 280 characters) on every question, included in the price.
escalatebooleanOptionalDefault false. true escalates in the same request (needs max_price_cents and an Idempotency-Key header).
max_price_centsintegerOptionalRequired iff escalate is true.

Unknown fields are rejected with 400 invalid_request and the offending path in error.field (for example questions[1].choices[0].weight), so content can never be silently dropped.

Input

The simplest input is a string. It is treated as one user message with one text part.

{
  "input": "Fictional offline AI evaluation. Source: Medication allergies are not documented. Model summary: The patient has no known medication allergies. Rubric: Preserve uncertainty; missing documentation is not a confirmed negative finding. Evaluate the summary, not patient care.",
  "questions": [
    {
      "type": "predicate",
      "name": "unsupported_claim",
      "instructions": "Does the summary make a clinical assertion unsupported by the source?"
    }
  ]
}

For structured cases, send an array of messages. Each message has role: "user" (or omit role), optionally type: "message", and content: a string or a nonempty array of parts.

PartFieldsNotes
input_texttextPlain text.
input_jsondataAn object, or an array of objects. Nested values can be any JSON. Up to 32 levels deep, 10,000 values, and 256 KiB in canonical compact form. Counted for billing as canonical JSON (see Pricing).
input_imageimage_url, optional detailA base64 data URL (data:image/png;base64,…) or an https:// URL. detail is low, high, auto or original.

Question instructions accept the same thing as message content: a string or an array of these three part types. Every content location follows the same screening, counting and image rules.

Images

  • At most 3 images per request, counted across input and question instructions together.
  • JPEG, PNG, HEIC, GIF or WebP. Up to 10 MB each and 40 megapixels.
  • http:// URLs and file_id references are rejected. Use https:// or a base64 data URL.
  • An HTTPS image is fetched once, when you submit. The physician sees exactly the image that was screened and previewed; changing the file at that URL later has no effect. URLs that resolve to private or internal addresses are refused.
  • Every image is normalized before anything else sees it: converted to a standard format, downscaled to at most 2560 px on the longest side, and stripped of all metadata, including location.
  • Each image adds a flat 500 billable tokens. Images do not count toward the 10,000-token text limit.
  • Images are screened for identifying content too: faces and visible text such as names, dates of birth or record numbers. Crop them out.

An image that can't be fetched or decoded returns 400 invalid_image naming the field.

Questions

Every question has a type, a name, and instructions. Names must be nonempty, unique within the request, at most 128 characters, and may not start with _appendix (reserved). Answers are matched to questions by name and returned in question order.

predicate

A yes/no question. No extra fields.

choice

Pick one of 2–255 choices. Each choice has a value and an optional description (up to 4,096 characters). Values are a nonempty string (up to 1,024 characters) or a boolean, and must be unique. Types are preserved: true and "true" are different values, and answers return the value with its original type.

{ "type": "choice", "name": "unsupported_claim",
  "instructions": "Does the model summary add a fact not supported by the source?",
  "choices": [ { "value": true }, { "value": false }, { "value": "insufficient_context" } ] }

score

Pick one of 2–10 ordered levels. Each level has a nonempty label (up to 1,024 characters) and an optional description. Levels are identified by their 0-based position.

AI answers

ai.answers has one entry per question, in question order.

{
  "ai": {
    "answers": [
      {
        "type": "predicate",
        "name": "unsupported_claim",
        "probability": 0.95
      },
      {
        "type": "choice",
        "name": "error_category",
        "choice": "unsupported_certainty",
        "confidence": 0.9,
        "probabilities": [
          {
            "value": "none",
            "probability": 0.03
          },
          {
            "value": "unsupported_certainty",
            "probability": 0.94
          },
          {
            "value": "insufficient_context",
            "probability": 0.03
          }
        ]
      },
      {
        "type": "score",
        "name": "error_severity",
        "score": 1.9,
        "confidence": 0.9,
        "probabilities": [
          {
            "value": 0,
            "label": "None",
            "probability": 0.02
          },
          {
            "value": 1,
            "label": "Minor",
            "probability": 0.06
          },
          {
            "value": 2,
            "label": "Material",
            "probability": 0.92
          }
        ]
      }
    ]
  }
}
TypeFields
predicateprobability: the model's probability, 0–1, that the answer is true.
choicechoice: the top value (original type); confidence; probabilities: {value, probability} per choice.
scorescore: the overall answer on the level scale, from 0 (lowest level) to the highest level index; confidence; probabilities: {value, label, probability} per level, where value is the 0-based level index.
refusalThe model declined to answer this question. name identifies the question. Other answers are unaffected.

AI values are model outputs and are returned without rounding. Treat any additional fields on an AI answer as optional, and handle refusal wherever you read an answer. The physician's answers use a different shape; see Physician Answers.

Differences from the structured decision format

The request follows a widely used structured decision format for typed questions. It is not full compatibility: the Decisions API accepts a documented subset, and is stricter in a few places. If you already produce requests in that format, check them against this list:

  • No model field. Appendix selects the model. Sending model is a 400.
  • User messages only. System, developer and assistant roles are rejected, not ignored.
  • No tool items. Tool calls, tool outputs and other non-message items are rejected.
  • Three part types. input_text, input_json and input_image. File references (file_id) and http:// image URLs are rejected.
  • Names are required and unique. Every question needs a nonempty name, so answers map back to questions without relying on position.
  • Up to 500 characters of text per question (its instructions; put long context in input), at most 3 images per request, at most 10,000 text and JSON tokens (and 100,000 characters), and at most 50 questions.
  • Appendix-only fields: reviewer, external_id, include_commentary, escalate and max_price_cents.
Unknown future values. Treat unknown status, knowledge.status, cancellation_reason and specialty values as possible. New values may be added without notice; new request fields are optional additions.