Attachment_1_Response_Form_Worksheet.docx

Attachment 1: Response Form Worksheet RFQ for Products, Software, and Services powered by Emerging Technologies Sourcing Event ID: 0000011726 What you need to do Complete and submit an online Response Form. This Response Form Worksheet is provided as a courtesy to firms who wish to draft their responses outside of the online Response Form. However, firms must submit their responses to these questions using our online Response Form; information submitted on this Worksheet will not be considered. Our Response Form was adapted from the City of San Jose’s AI Factsheet, and firms are encouraged to leverage their responses to the Factsheet when completing this Response Form.

How to answer these questions Firms should answer each question based on their current product; future engagements with the City will be governed by the terms in the RFQ solicitation document and Attachments (“Solicitation”).

· For example, for question 6, we ask firms to tell us about your training data. Firms should answer the question based on work you have completed to date on your current product. However, if selected for a short-term contract, the City will require firms to adhere to our training data requirements listed in the Technical Requirements attached to the Solicitation as Attachment 4.

How will this information be used?

Information submitted will be used to validate that firms meet the Minimum Qualifications listed in the Solicitation. In addition, the City reserves the right to consider the information submitted as part of subsequent procurements (mini-RFPs) from this RFQ Pool, including by scoring firm responses to the questions below.

Sections of Response Form Worksheet

1) Section 1: Firm Information

2) Section 2: Product Information

3) Section 3: Previous Deployments

4) Section 4: Acknowledgement of City requirements

Section 1: Firm Information Please provide the following information about your firm.

1. Firm Name and Contact Info. Provide your “Doing business as” Name and Legal Name, if different, as well as point of contact, email, and phone contact information.

Provide your response here.

Section 2: Product Information Please provide the following information about your product. The City will not review or consider attached general marketing materials or other documentation provided by firms as a substitute for answering these questions.

2. Product Name(s). Name of the AI product or suite of products.

Provide your response here.

3. Product Description. Briefly describe your AI product or suite of products.

Provide your response here.

4. Product Purpose. What function does the AI system perform, and for what purpose? If the system performs multiple functions, list each discretely and reference below. For features that are configurable, please describe all configuration options and default settings.

Provide your response here.

5. Product Functionality. Please select all that apply.

☐ Document management (e.g., parsing, content extraction, classification, etc.)

☐ Notetaking (e.g., speech to text, meeting notes, etc.)

☐ Video generation for e-learning or similar ☐ Language translation ☐ Chatbots and conversational AI, including website chatbots (for public and internal facing use) ☐ Enterprise knowledge assistant/agentic assist ☐ Image and video analysis ☐ Predictive analytics and decision support tools ☐ Anomaly detection ☐ Procurement ☐ Report writing and generation ☐ AI bias and performance monitor ☐ Transportation operations intelligence ☐ Other _______________________

6. Intended Domain. What domain is the AI system intended to be applied in?

Provide your response here.

7. Training Data. How was the AI system trained? What data was used? How often is data added to the training set? Was all training data legally obtained and its use fully licensed?

Provide your response here.

8. Test Data. Describe what data was used to test system performance. Under what conditions has the system been tested?

Provide your response here.

9. Model Information. Provide a general description of the model(s) used (e.g., large language model, transformer, deep learning, supervised learning, built on an existing open-source model, computer vision).

Provide your response here.

10. Update Procedure. In general, how often is the model(s) updated for users? Will the user have a choice in moving to the updated model or staying on the current model? What documentation is available for new versions of the model?

Provide your response here.

11. Inputs and Outputs. What are the inputs to the AI system? What are its outputs? What interfaces and integrations are supported?

Provide your response here.

12. Performance Metrics. What are the performance metrics? What is your current level of performance on these metrics? How can the user monitor performance when deployed?

Provide your response here.

13. Bias. What biases does the tool exhibit and how does it manage that bias? This can include but is not limited to biases on human factors such as gender, race, socioeconomic status, disability, culture, age, or other protected classes, or biases on general factors such as a sampling bias, survivorship bias, detection bias, or observer bias.

Provide your response here.

14. Robustness. How does the AI system handle outliers? Do overwritten decisions feed back into the system to help calibrate it in the future?

Provide your response here.

15. Optimal Conditions. What conditions does the model perform best under? Are there minimum requirements for the quantity of records/observations?

Provide your response here.

16. Poor Conditions. What conditions does the model perform poorly under? What are the limitations of the AI system? What kinds of errors can it make (e.g., hallucinations) and what conditions make those errors more likely?

Provide your response here.

17. Explanation. How does the AI system explain its outputs? Are the outputs of the AI system understandable by subject matter experts, users, impacted individuals, etc.?

Provide your response here.

18. Data Protection. List data protection frameworks you comply with (e.g., NIST 800-53, NIST AI RMF) and certifications (e.g., SOC II, HIPAA, FERPA, CJIS).

Provide your response here.

19. Environmental Impacts. This can include, but is not limited to, energy and water consumption for training and/or per use of the model, sources of energy and water for your solution, and your organization’s approach to reducing environmental impact.

a. For energy consumption, report in megawatt-hours (MWh).[footnoteRef:1] [1: If you are unsure how to conduct energy measurements, see “A Beginner’s Guide to Power and Energy Measurement and Estimation for Computing and Machine Learning” published by the National Renewable Energy Laboratory and Intel.]

b. For water consumption (in thousand liters), include the type of water (i.e., drinking water or reclaimed water) used in training and deploying the model (if the on-site water usage is unknown, use "1 L/kWh" as a default value).[footnoteRef:2] [2: If you are unsure how to conduct water impact measurements, see “Making AI Less 'Thirsty': Uncovering and Addressing the Secret Water Footprint of AI Models” published in Communications of the ACM.]

Provide your response here.

20. Ongoing Monitoring. How is the AI system monitored to identify any problems in usage? Can outputs (recommendations, predictions, etc.) be overwritten by a human, and do overwritten outputs help calibrate the system in the future?

Problems in usage can include false negatives, false positives, bias, hallucinations, and human-reported quality issues (such as poor translations or poorly generated images).

Provide your response here.

21. Independent Evaluation. Has your firm or an independent party conducted a study on the bias, accuracy, or disparate impact of the system? If yes, can the City review the study? Include methodology and results. This can include bias impact reports, algorithmic impact reports, or others.[footnoteRef:3] [3: See “Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms” by the Brookings Institute for an example bias impact report template.]

Provide your response here.

22. Accessibility. How does your solution accommodate a variety of users, such as users with disabilities?

Provide your response here.

23. Responsible AI Strategy. Please share any additional relevant information, links, or resources regarding your firm’s responsible AI strategy.

Provide your response here.

Section 3: Previous Deployments All Proposers must provide references for at least two (2) past or continuing clients – either governmental, commercial, or institutional – where Proposer has successfully deployed its solution in a production setting. City may verify your listed references, so you must ensure that they are reachable and expect to be contacted. If City is unable to verify at least two (2) of your provided references, it may result in your proposal being deemed non-responsive.

Name of Client #1

Address (street, city, state, zip)

Contact Name

Phone #

Email

Period of deployment

(MM/YY – MM/YY)

Brief Description of Client Need and Your Solution

Name of Client #2

Address (street, city, state, zip)

Contact Name

Phone #

Email

Period of deployment

(MM/YY – MM/YY)

Brief Description of Client Need and Your Solution

Section 4: Acknowledgement of City requirements Failure to acknowledge the below will result in your proposal being deemed non-responsive.

Acknowledgement of City requirements. My firm’s submittal of a response to this Request for Qualifications explicitly stipulates acknowledgement of the following.

☐ My firm agrees to all terms found in the Solicitation as well as in any and all addenda.

☐ My firm acknowledges that, if awarded a “Short-Form Contract” as described in this Solicitation, we substantially agree to all terms and conditions specified in Attachment 2, “Short-Form Contract Terms.”

☐ My firm acknowledges that if awarded a “Long-Form Contract” as described in this Solicitation, we substantially agree to all terms and conditions specified in Attachment 3, “Long-Form Contract Amendment Terms,” including terms related to indemnity, insurance, termination, and limits of liability. We understand that the ability to negotiate terms and conditions will be limited.

☐ My firm acknowledges that if awarded a “Short-Form Contract” or “Long-Form Contract” as they are described in this Solicitation, we will satisfy all corresponding requirements listed in the Technical Requirements in Attachment 4 (“Technical Requirements”), which includes requirements related to sandboxes, prototype testing, security and privacy, data, cybersecurity, AI transparency, and public-facing content.

☐ My firm acknowledges that failure to enter into a contract with the City in a timely manner may result in the city removing my firm from the RFQ Pool.

Attachment 1: Response Form Worksheet Page 1 of 6 Sourcing Event ID: 0000011726