Category: Charts
A reliability diagram plotting predicted probability against observed accuracy for one or more series, with a dashed diagonal reference line for perfect calibration and optional confidence-interval bars per point. Point radius can grow with sample size (`n`). Answers whether a model's own confidence scores are trustworthy, not just whether its answers are right.
| Field | Type | Required |
|---|---|---|
| title | string (optional, e.g., 'Jev confidence calibration') | optional |
| x_label | string (optional, default 'Predicted probability') | optional |
| y_label | string (optional, default 'Observed accuracy') | optional |
| series | list of {label: string, color: string (optional hex, auto-assigned otherwise), points: list of {predicted: number 0-1, observed: number 0-1, n: integer (optional sample size, grows point radius), ci_low: number, ci_high: number}} | optional |
{
"type": "calibration_plot"
}{
"identifier": "urn:air:a2uicatalog.ai:atom:calibration_plot",
"displayName": "Calibration Plot",
"type": "application/vnd.a2ui.atom+json",
"url": "https://a2uicatalog.ai/atoms/calibration_plot/",
"capabilities": [
"web"
],
"description": "A reliability diagram plotting predicted probability against observed accuracy for one or more series, with a dashed diagonal reference line for perfect calibration and optional confidence-interval bars per point. Point radius can grow with sample size (`n`). Answers whether a model's own confidence scores are trustworthy, not just whether its answers are right.",
"representativeQueries": [
"show a predicted-vs-observed probability calibration chart with confidence intervals",
"a reliability diagram plotting predicted probability against observed accuracy for one or more serie",
"render a calibration plot"
]
}The renderer is an open-source Google Apps Script web app. Deploy your own instance in 4 commands — you own the URL, no dependency on the demo endpoint.
git clone https://github.com/a2uicatalog/a2ui
cd a2ui/apps-script-surface/gas-wired-renderer
clasp push && clasp deploy