17-Appendix M WIP.pdf

Appendix M: Requirements for AI Systems

This Addendum defines special requirements agreed to by the City of Dallas and Contractor regarding the AI system and/or subsystem provided as part of the Contract.

This Addendum governs over any contrary license terms and the City of Dallas will not agree to any terms that conflict with the Addendum. Failure of the Contractor to comply with the terms of this Addendum shall constitute a material breach of the Contract.

1. Definition of AI System

AI definitions are codified in Section 551.001(2) of the Texas Business & Commerce Code as amended by TRAIGA 2.0.

2. Guiding Principles

The Contractor shall demonstrate that the AI system and its usage, deployment, and maintenance as it pertains to the services outlined in this agreement do not conflict with TRAIGA and do not conflict with the City of Dallas's AI Policy and Principles therein.

3. Risk Mitigation

The Contractor represents that the AI system is suitable for its intended use by the City of Dallas and has been developed and will perform in a manner that is in compliance with all applicable laws and regulations.

The Contractor shall maintain, follow, and provide, upon request, a copy of their Risk Management Plan that complies with accepted risk management frameworks, such as the NIST AI Risk Management Framework1, ISO 420012, ISO 47001.

The Contractor shall work with the City of Dallas to evaluate and minimize risks posed by the AI system.

AI Incident Response In the event of an AI incident, at the request of the City of Dallas, the Contractor shall thoroughly investigate their systems of any suspected AI incident and immediately report findings to the City of Dallas. An “AI incident” is an alleged harm or near harm event to people, property, reputation, technical integrity of the environment where an AI system is implicated. Examples of AI incidents include providing false information, copyright

1 https://www.nist.gov/itl/ai-risk-management-framework 2 https://www.iso.org/standard/81230.html https://www.nist.gov/itl/ai-risk-management-framework https://www.iso.org/standard/81230.html infringement, generating harmful bias, system misuse, exposure of sensitive information, and liability risk.

Remediation At the City of Dallas’ request, the Contractor will immediately discontinue the use of any AI system involved in providing services to the City of Dallas. If the City of Dallas, in its sole discretion, determines that the AI system is in violation of one or more of the requirements enumerated in Section 4: Requirements for Contractors when Operating AI System(s), or that the Contractor does not promptly resolve an AI incident, or that the system does not adequately supports the City of Dallas’ commitment to the City of Dallas's AI Policy and Principles, then the City of Dallas will provide the Contractor with notice that they have 30 calendar days to promptly assess and resolve the issue. Potential methods to address such issues include, without limitation, changing the behavior of the AI system or subsystem;

supplementing the system or subsystem to achieve the necessary outcomes; replacing the system with a non-AI system that meets the City of Dallas’ needs; or limiting the function of the AI system or subsystem. After 30 calendar days, the Contractor must provide evidence that the AI system is adequately fixed and ready for re-deployment, or that the AI system is not suitable for use.

4. Requirements for Contractors AI System(s)

To the extent permissible by law, the Contractor shall adhere to the following requirements in the course of doing business with or for the City of Dallas:

1. Updates: Contractor attests that the previously completed or most recently submitted AI FactSheet accurately represents the AI system (See Information & Technology Services Enterprise Architecture: AI FactSheet). The contractor shall provide prior notice for system upgrades that substantially impact the services provided to the City of Dallas. Contractor commits to update the AI FactSheet on an annual basis and within 30 calendar days of any change to the AI system that substantially impact the services provided to the City of Dallas. Example of a substantial impact includes but is not limited to changes to sensitive or protected information used in the system, compatibility with the City of Dallas’s other systems, or decisions that impact the rights or safety of individuals. In the event of a change with a substantial impact, vendor will provide the City of Dallas opportunity to stay on the existing system version at the City of Dallas’s request. Any updates made to the AI system that render the AI system unserviceable to the City of Dallas to provide services defined in the agreement as determined by the City of Dallas may be cause for termination by the City of Dallas.

2. Data ownership: The City of Dallas retains ownership of any data it provides to the Contractor, as well as data collected or generated specifically on behalf of the City of Dallas during the course of the Contract. The Contractor retains ownership of all other data it collects or generates, including data created for the Contractor's general business purposes. The City of Dallas has the unrestricted right to use its https://dallascityhall.com/departments/ciservices/Pages/enterprise-architecture.aspx https://dallascityhall.com/departments/ciservices/Pages/enterprise-architecture.aspx owned data for any purpose. Ownership of such data shall not be transferred to the Contractor or any third party without prior written consent from the City of Dallas.

3. Data portability: At the City of Dallas's request, and at no additional cost, the Contractor shall provide all data owned by the City of Dallas in a format that is easily usable by the City of Dallas and, to the extent technically feasible, in a structured, commonly used, machine-readable format. The Contractor is not obligated to provide such data more than twice in a 12-month period.

4. Data Authorized use: Data owned by the City of Dallas cannot be used for other commercial applications without written consent from the City of Dallas.

Anonymized data may not be used freely by the contractor.

5. Data rights: Data owned by the City of Dallas that contains sensitive or protected information must be able to be deleted, edited, or anonymized at the request of the City of Dallas or the data subject.

6. Performance: Provide the City of Dallas or entity working on behalf of the City of Dallas with the means to monitor the performance, including the accuracy, of the AI system it uses and report this accuracy to the City of Dallas. This may include, but is not limited to, the false positive rate, the false negative rate, the true positive rate, the average percentage error, the mean-squared error, and human judgement scores.

7. Algorithmic bias: Contractor will provide the City of Dallas with evidence that demonstrates that bias present in the AI system is effectively managed for the context in which it will be deployed. Contractor shall provide information describing in detail how bias is assessed.

8. Human oversight: Provide the City of Dallas the means for a human to evaluate and override outputs of the AI system. The human evaluator must be able to override the outputs of the AI system and take precedence over all outputs.

9. Explainability: Provide the City of Dallas with an explanation of how the AI system generates outputs, including what factors influence the system’s decisions, rule-based logic, training data sources, and probability-based decisions. The City of Dallas holds the right to communicate its general usage of the AI system and explain its decision-making processes to the public. Additionally, in the event of an AI Incident, the Contractor shall provide a detailed, case-specific explanation of how the AI system generated the output or decision implicated in the incident. This explanation should include contributing factors, the system’s decision-making process, and any relevant data or models used.

10. Notice: If required by the City of Dallas, provide written notice of the usage of the AI system to data subjects and/or end-users, preferably at the point of service.

11. Process: Contractor shall comply with existing local, state, and federal law for data access and decision appeals related to the use or operation of the system.

12. Ongoing Monitoring: Contractor shall regularly monitor the performance of the AI system to detect and rectify system behavior that violates any of the requirements in this section. Contractor shall promptly communicate the discovery of system behavior that violates any of the requirements in this section to the City of Dallas, including the potential impact to services.

13. Training: Ensure that appropriate training is available to City of Dallas staff who may operate the AI system, which may include how to:

1. Protect sensitive or personal information

2. Mitigate harmful algorithmic bias

3. Promote optimal performance

4. Report system errors and

5. Maintain service delivery if the AI system fails, to the extent possible.

14. Auditing: The City of Dallas retains the right to observe, assess, or audit any relevant work processes, services, or documents in the course of doing business with the City of Dallas to confirm that the Contractor (and any relevant sub-contractors) is complying with this Policy. Contractor shall provide access to information, documentation, and personnel required to complete this audit at no additional cost to the City of Dallas.