Request Software or Technology

Systems posing extreme risks to public welfare, safety, or rights that cannot be mitigated. They may violate fundamental rights (e.g., unlawful surveillance, social scoring) or fail to align with State values. These systems are banned entirely, regardless of data classification, as no combination of safeguards can offset the risk they pose. Use cases that result in violations of law or core civil liberties are not permissible.

• Data Classification Type(s): No Relevant Data Classification
  • There is no corresponding data classification that suggests this risk label. Even if the data in question might normally be classified at Level 1, 2, 3, or 4, the system itself is not permissible due to the nature of its risk, rather than the sensitivity of the data.
    For instance, a system that attempts to track or socially score individuals en masse would be “unacceptable,” even if it only accessed public (Level 1) data. The risk category overrides any data classification considerations. ​
• ​​Use Case Example(s)
  • Unlawful Mass Surveillance: For example, an AI tool that attempts to track constituents’ every movement via facial recognition in real time without oversight or statutory authority.​
  • Uncontrolled Social Scoring: A system that ranks or penalizes constituents’ daily behavior, akin to certain types of “social credit,” contradicting fundamental principles of fairness and privacy.​
    (​Even if such a system used largely public data, it would remain “unacceptable” based on the AI policy’s ethical and legal prohibitions.) ​

AI systems that significantly affect individuals or critical government operations. Often influence outcomes related to health, safety, law enforcement, eligibility for essential services, privacy, financial or legal rights, or other high-impact areas (civil rights, civil liberties, equal opportunities, access to critical resources, or privacy).

• ​Mandatory Safeguards
  • ​Comprehensive Algorithmic Impact Assessment (AIA) before deployment.​
  • Risk mitigation measures (e.g., bias testing/correction, cybersecurity hardening, etc.).​
  • Ongoing monitoring, with human-in-the-loop oversight for critical decisions.​
  • Aligns with NIST AI RMF best practices (mapping, measuring, mitigating, and managing risk continually).​​
• ​Data Classification Type(s): Likely Level 4 (Restricted) or Level 3 (Confidential)​
  • Level 4 (Restricted) data involves severe impacts if disclosed (e.g., Criminal Justice Information (CJI), Federal Tax Information (FTI)). Unauthorized access could cause irreparable harm, may carry legal or criminal consequences, and is typically protected by strict laws (IRS Publication 1075, CJIS Security Policy, etc.).​
  • Level 3 (Confidential) data includes sensitive information (PII, PHI, financial/student data), which is protected by law from disclosure. Unauthorized access can severely harm individuals or the State and requires safeguards such as encryption at rest/in transit. ​AI systems that process or make critical decisions using these high-sensitivity data levels fit squarely into High-Risk. ​​
• ​​​Use Case Example(s)
  • AI for Eligibility Determinations: A tool that decides whether individuals qualify for social services (e.g., Medicaid, housing assistance). It uses Confidential (Level 3) data like PII and financial records. Because erroneous or biased decisions could seriously impact constituents, it is “High-Risk” and requires a full Algorithmic Impact Assessment (AIA).​
  • Law Enforcement Investigative AI: An AI system that taps into Restricted (Level 4) data (e.g., Criminal Justice Information) to recommend suspect lists or inform sentencing guidelines. Such usage has high stakes for individual rights and must be thoroughly vetted, tested for bias, and subject to human oversight.​
  • Financial Lending/Underwriting Tool: An AI that sets loan terms for constituents or businesses, using robust sets of personal/financial data. A misclassification, bias, or data breach could cause serious harm, making it High-Risk.​​

AI systems with moderate or low overall impact that do not autonomously decide critical outcomes for individuals (e.g., no direct control over law enforcement, healthcare coverage, or major financial decisions – humans need to be the decision maker of any AI output). Typically used to improve internal efficiency or enhance customer service with minimal chance of harm if errors occur.

• ​Mandatory Safeguards
  • Light-touch requirements: transparency (e.g., disclose use of AI in chatbots), basic testing for fairness/accuracy.​
  • If usage expands or new risks emerge, reclassify as High-Risk.​​
• Data Classification Type(s): Primarily Level 2 (Protected / Internal Use Only)​
  • Level 2 data might include internal agency documents (draft analyses, budget proposals, memos) or other data not meant for broad public release but not legally restricted. Unauthorized disclosure could result in some form of negative impact, but not on par with potential harms related to PII or CJI.
  • Could occasionally involve Level 1 (Public) data if the system uses openly available information but still requires moderate protective measures (e.g., an internal tool that references public stats but must remain behind an internal firewall).​
  • ​In general, Limited-Risk systems do not handle highly confidential or restricted data.
• ​​​​​Use Case Example(s)
  • AI-Driven FAQ Chatbot (External): A virtual assistant to help the public find information on agency services. It may use a combination of Level 1 public data and some Level 2 internal knowledge base. Impact is moderate—errors might cause confusion, but not irreparable harm.​
  • Internal Analytics Dashboard: Aggregates departmental performance metrics from Protected/Internal (Level 2) data (like draft budget files) to present dashboards for leadership. Misclassification could cause moderate risk, but it typically doesn’t impact individual rights in a critical way.​
  • Customer Service Triage Tool: Routes inquiries or complaints internally based on fairly routine categories. The data is mostly administrative and does not involve legally protected personal information.​

AI applications that pose negligible risk, usually embedded in standard office software or used for non-critical, purely internal tasks (e.g., spam filters, grammar correction). They do not impact individual rights or major government processes if they malfunction.

• ​Safeguards ​
  • No special AI-specific approvals beyond standard IT/security reviews.​
  • Still recommended to track/catalog these tools so that if they malfunction, they can be reassessed for potential reclassification.​
• ​Data Classification Type(s): Most Commonly Level 1 (Public) Data, Possibly Low-Sensitivity Level 2
  • Level 1 (Public) data: Data that can be freely shared without negative consequences (e.g., open datasets, publicly available information).
  • If the tool processes Level 2 (Internal Use) data, it is typically in ways that present extremely low risk (e.g., scanning a publicly available document for grammar suggestions).​​
  • ​Minimal-risk AI does not rely on sensitive personal data or restricted data.​​​​
• ​​Use Case Example(s)
  • ​Email Spam Filter: Uses low-level AI to sort phishing from legitimate email. Even if it occasionally flags a legitimate message, the risk is negligible. Operates on routine data that is not highly sensitive in bulk form.​
  • Spell Checker / Grammar Tool: An AI embedded within a word processor that checks for mistakes. The data involved is primarily open or low-sensitivity text, meaning minimal-risk if misclassified or leaked.​
  • Automated Meeting Scheduler: An AI that finds open calendar slots among staff. The data is mostly staff names and times, not restricted or confidential. If it fails or leaks, the harm is minimal.​

AI synthetic products from AI systems would be governed by the State of MD policies around record management. A record is any documentary material created or received by an agency in connection with the transaction of public business. Agencies will need to consult their record retention schedules regarding retention and disposal requirements. Each agency is required to have a Records Officer from among its executive staff who is responsible for developing and overseeing the agency’s records management program and serves as a liaison to the Maryland Archives, and they should be consulted as well.

This framework will integrate with existing risk management processes and provide tools for evaluating AI-specific risks (like bias, security vulnerabilities, reliability issues). As part of monitoring, you should revisit this assessment and risk classification to ensure there are no changes from your initial assessment.

Set up processes to continuously monitor the AI system’s performance and behavior.

• ​Performance Metrics: Track metrics that indicate if the AI is working as intended. For example, the accuracy of predictions, the number of errors or exceptions, response time, uptime, etc., depending on the system. A drift in these metrics could signal a problem (e.g., the model’s accuracy might degrade over time as real-world data evolves).
• Outcome Auditing: Regularly audit the outcomes of the AI for fairness and correctness. If it’s supporting decision making, sample those decisions periodically and ensure they align with policy and there’s no systemic bias. This might be done by an internal audit team or in some cases by an external auditor or an AI ethics board. Bias mitigation is not a one-time task; continuous auditing helps catch emergent biases or unintended discrimination.
• User Feedback: Provide channels for users (employees or public users) to report issues with the AI. For instance, if an employee thinks the AI recommendation system is making odd suggestions, they should know how to flag this. Public-facing AI in particular should have a feedback mechanism, and your agency should monitor if there are complaints or questions coming in related to the AI’s decisions.
• Error Logs and Incident Response: Monitor error logs or any fail-safe triggers. If for example the AI system has a fallback when it’s unsure (e.g., escalates to a human or refuses to answer), log those events and review them. Establish an incident response plan for AI issues: if something potentially harmful or unethical is detected (like the AI made a privacy violation or a biased decision), have a plan to immediately intervene (disable the AI function if needed), analyze the issue, and inform the necessary oversight bodies. If individuals might have been negatively impacted by a high-risk AI, DoIT should be notified and the Agency with DoIT’s assistance as needed will guide affected individuals — so loop in DoIT as part of your incident protocol for major issues.
   

Maryland law specifically requires agencies using high-risk AI to conduct regular impact assessments of those systems​. This should be done at least annually - and can be completed by re-evaluating the system via the Algorithmic Impact Assessment.

Be ready to adjust or even deactivate the AI if risks become unacceptable. Managing AI risk is an active process:

• If monitoring shows performance issues, plan a model update or retraining. (Coordinate with the vendor if it’s their product – ensure they provide timely updates or patches.)
• If an audit reveals bias, take corrective action: this could be data augmentation, tweaking decision thresholds, or adding a rule-based layer to counteract the bias. For example, if a recruiting AI is scoring female candidates lower, you might adjust the model or introduce a step to ensure gender-neutral evaluation criteria.
• For security risks (e.g., adversarial attacks or data leaks through the AI), work with your cybersecurity team and DoIT’s Office of Security Management. As AI systems can introduce new attack surfaces, ensure they are part of your cybersecurity audits. 

Maintain transparency in your AI operations as part of risk management:

• ​Continue to update the public AI inventory with current information about the system (see the Transparency section). If the system changes significantly (new version or new uses), that should be reflected.
• Be prepared to share information about the AI’s performance and governance with oversight entities. The AI Subcabinet or legislative committees may request updates or conduct reviews. Having your monitoring data and assessment reports organized will make these interactions smoother and demonstrate your agency’s accountability.
• If appropriate, publish summary results of your AI’s audits or impact assessments on your agency website or in public reports.