Tools · LEADERSHIP

Set limits for AI in health care

Describe an AI task, its limits and the human decisions it requires

Go to the questions →About 20–30 minutes for a first pass · Every question is optional

Describe one proposed use of AI in health care. Record what the software would do, what evidence you have, where it must stop and who would remain responsible. Use this to prepare questions for the people making the decision. A fictional clinic is considering software that drafts appointment summaries. The software may propose text, but a clinician must check the source and approve the note. If information is missing, the software must show the gap rather than invent a finding. That is a proposed boundary to test, not proof that a product meets it.

Before you type anything

What happens to what you type

This worksheet does not send, save, or track anything you type. Entries stay in the page until you clear it or leave. Your own browser, keyboard, or extensions may still process what you type, so do not enter anyone's name or identifying details. The optional Copy button puts your summary on your device clipboard, where other software on your device can read it.

What this tool can't do

Patient care, diagnosis, treatment selection, triage, risk prediction, or an individual clinical decision
Regulatory classification, legal advice, privacy impact assessment, security assessment, procurement approval, or clinical authorization
A product score, recommendation, certification, endorsement, or claim of validation
Entering patient information, personal information, proprietary prompts, confidential incidents, source code, or security details

When to stop and use another route

Do not use this worksheet to authorize deployment. Unresolved evidence, accountability, privacy, safety, regulatory, licensing, or local-policy questions remain unresolved after the worksheet is complete.
Stop the proposed use and route it to the appropriate accountable clinical, privacy, security, legal, regulatory, procurement, and organizational authorities when its boundaries cannot be established.

The questions

The whole exercise stays visible, and every question is optional. Start with the question that catches something useful, leave anything blank, and stop when the exercise stops helping.

Describe the proposed use

A system cannot be assessed in the abstract; start with the user, setting, and exact job.

What exact task would the software perform?
Use a specific action, such as drafting an appointment summary. “Improve care” does not identify a task.
0 / 2,000 characters
Who is the intended user, and what authorization or competence must that person have?
Distinguish operators, recipients, reviewers, and the person accountable for the decision.
0 / 2,000 characters
In what exact setting and workflow is the task intended to occur?
Name the jurisdiction, service context, timing, handoffs, and meaningful exclusions without identifying an organization.
0 / 2,000 characters

Record the decisions and information involved

Make the affected decision, the human-only boundary, the inputs, and their provenance explicit.

What decision or action could the output affect?
Include indirect effects such as what gets noticed, prioritized, documented, deferred, or omitted.
0 / 2,000 characters
What decisions, claims or messages must the software never make on its own?
For example: it must not add an unverified diagnosis or send a clinical recommendation without the required human decision.
0 / 2,000 characters
What information would the software use, and what could be left out?
Record types of information, not actual records. Include how it is collected, combined, stored, accessed and reused.
0 / 2,000 characters
How would a user check where important information came from?
Record available source links, dates, transformations and software versions, as well as anything that cannot be checked.
0 / 2,000 characters

List the gaps and possible harms

A polished output must not hide missing evidence, untested settings, abstention rules, or failure consequences.

How are missing, conflicting, low-quality, or stale inputs made visible?
Describe uncertainty honestly. Do not let a polished output hide absent or contradictory evidence.
0 / 2,000 characters
When must the software refuse a task or stop?
List conditions that could be detected, such as missing information, an unsupported patient group or the absence of the required human reviewer.
0 / 2,000 characters
What has actually been tested, with whom and in which setting?
Record the study or test, comparator and result. Separate accuracy of the software from the quality of decisions made by people using it. List untested uses.
0 / 2,000 characters
What is the worst foreseeable harm if the output is wrong, late, biased, persuasive, or unavailable?
Consider omission, automation bias, inequitable performance, privacy harm, delay, over-reliance, and failure of the surrounding workflow.
0 / 2,000 characters

Assign responsibility for checks and changes

Name who can inspect, disagree, verify, intervene, and stop the system across its lifecycle.

Which authorized human remains responsible, and what can that person actually see and do?
Name the role, review point, authority to disagree, time available, escalation route, and accountability—without naming a person.
0 / 2,000 characters
What requires independent verification before any action is taken?
Specify source checking, second review, testing, audit, or other controls proportionate to the affected decision.
0 / 2,000 characters
Who would handle incidents, updates, permissions and stopping the service?
List the responsible roles, when they must act, how use can be stopped and where their decisions are recorded. Leave a gap visible when no role has been assigned.
0 / 2,000 characters

What you have so far

Builds as you type

Judgment-boundary note

Your own words, kept in order. It gives no classification, recommendation, validation finding, or authorization.

SOFTWARE MAY ORGANIZE: No entry yet
HUMAN JUDGMENT REMAINS: No entry yet
EVIDENCE NEEDED: No entry yet
ABSTAIN / STOP: No entry yet

This is a structured copy of your own entries. The tool does not score, interpret, diagnose, predict, or recommend.

Keep it or clear it

There is no submit button. Use Print or Copy if you choose to keep a copy, then Clear before leaving.

Before you use this to make a decision

Take the unanswered questions to the people responsible for the proposed use. Completing this worksheet does not authorize a deployment, determine regulatory status or establish that a system is safe or effective.

Sources and limitations

What the evidence does not establish

This worksheet produces no regulatory classification, validation finding, risk score, procurement advice, clinical recommendation, or authorization.
Official requirements depend on the product, intended use, jurisdiction, data flows, user population, deployment context, and current law and policy.
Evidence about a model alone does not establish the safety or effectiveness of the human–AI team in the intended workflow.
Completing every prompt is not proof that the answers are accurate, independently verified, current, or accepted by the accountable authorities.
Guidance and system performance can change; this review is designed for a six-month interval and after every material change or incident.

The sources behind the structure

Health Canada — Pre-market guidance for machine learning-enabled medical devices (2026) ↗Current Canadian guidance on intended use, risk, data, testing, validation, transparency, and monitoring.
Health Canada, FDA and MHRA — transparency guiding principles for ML-enabled medical devices ↗Covers intended use, human–AI team performance, data, bias, limitations, failure modes, and lifecycle information.
Canadian privacy regulators — principles for responsible and privacy-protective generative AI ↗Addresses legal authority, necessity, proportionality, transparency, safeguards, retention, accountability, and human review.
Pan-Canadian AI for Health Guiding Principles ↗Canadian principles for equity, privacy, security, safety, oversight, accountability, robust data, and Indigenous data sovereignty.
IMDRF — Software as a Medical Device: Clinical Evaluation ↗Framework for valid clinical association, analytical validation, and clinical validation of SaMD.

← All tools