7-AI Factsheet Sample 9.8.26.pdf

AI Fact Sheet

Please provide details regarding your Artificial Intelligence (AI) system or product or component of the product by filling out the Factsheet template below, if any. “AI” is defined as any machine-based system that can make decisions, recommendations, or predictions based on human-defined objectives. This information will be kept by the City of Detroit and may be publicly available. An example of a completed FactSheet:

AI Fact Sheet

Vendor Name

AI Name(s)

Overview Brief summary of the AI.

Purpose What function does the AI perform, and for what purpose? If the AI performs multiple functions, list each discretely and reference below. For features that are configurable, please describe all configuration options and default settings.

Intended Domain What domain is the AI intended to be applied in?

Training Data How was the AI trained? What data was used? How often is data added to the training set? Was all training data legally obtained and its use fully licensed?

Test Data What data was used to test AI performance? Under what conditions has the AI been tested?

Model Information General description of the model(s) used (e.g., large language model, transformer, deep learning, supervised learning, built on an existing open-source model, computer vision)

Update procedure In general, how often are the models updated for users? Will the user have a choice in moving to the updated model or staying on the current model? What documentation is available for new versions of the model?

Inputs and Outputs What are the inputs to the AI? What are its outputs? What interfaces and integrations are supported?

Performance Metrics

What are the performance metrics? What is your current level of performance on these metrics? How can the performance be monitored in the deployment environment?

Bias Was the AI evaluated for bias? How was the tool evaluated for the bias? What biases does the AI exhibit and how does it handle that bias? This can include but is not limited to biases on human factors such as gender, race, socioeconomic status, disability, culture, age, or other protected classes, or biases on general factors such as a sampling bias, survivorship bias, detection bias, or observer bias.

Robustness How does the AI handle data outliers? Do overwritten AI output feed back into the AI to help calibrate it in the future?

Optimal Conditions What conditions does the AI perform best under? Are there minimum requirements for the quantity of records/observations?

Poor Conditions What conditions does the AI perform poorly under? What are the limitations of the AI? What kinds of errors can it make (e.g., hallucinations) and what conditions make those errors more likely?

Explanation How does the AI explain its predictions? Are the outcomes of the AI understandable by subject matter experts, users, impacted individuals, and others?

Jurisdiction-specific Considerations

Please describe any considerations relevant to local, state, industry, or other specific jurisdictional regulations.

Algorithmic Impact Assessment Questionnaire How is the AI monitored to identify any problems in usage? Can outputs (recommendations, predictions, etc.) be overwritten by a human, and do overwritten outputs help calibrate the AI in the future?

Problems in usage can include false negatives, false positives, bias, hallucinations, and human-reported quality issues (such as poor translations or poorly generated images).

How is bias managed effectively? This can include ways to monitor bias, or abilities to toggle parameters to change observed bias in the model.

Have the vendors or an independent party conducted a study on the bias, accuracy, or disparate impact of the AI? If yes, can the Agency review the study? Include methodology and results.

This can include bias impact reports, algorithmic impact reports, or others.2

How can the Agency and its partners flag issues related to bias, discrimination, or poor performance of the AI?

This can include ways to report inaccurate or concerning output /classifications made by the AI, or ways to retroactively review past AI actions.

How has the Human-Computer Interaction aspect of the AI tool been made accessible, such as to people with disabilities?

Has it been assessed against any usability standards, and if so, what was the result?

Please share any relevant information, links, or resources regarding your organization’s responsible AI strategy.

URL to any broad AI policy or strategy.