Attachment K.8 DRD 5- LLM Transparency.pdf

Rev.: 03/2026

1. DRD Title: Large Language Model (LLM) Transparency

2. DRD No.: 005 3. Data Type: 2 4. OPR: OCIO

5. Solicitation No.:

80GSFC27R0018

6. Contract No.:

7. Date Issued: March 2026 8. Date Revised: N/A 9. DRD Category:

Technical ☐

Administrative ☒

S&MA ☐

10. Description/Use: To ensure that large language models (LLMs) and other artificial intelligence (AI) models comply with Unbiased Artificial Intelligence (AI) Principles.

11. Distribution: Initial submission with proposal. All subsequent deliverables required by this DRD must be submitted electronically to the contracting officer, contracting officer’s representative, and NASA’s Chief Artificial Intelligence Officer (CAIO) at agency-caio@mail.nasa.gov, or as otherwise directed by the contracting officer.

Initial Submission:

The initial submission of this DRD is material to eligibility for award.

Submission Frequency: Content section 1 (Minimum Threshold for LLM Transparency) and if applicable, section

2 (Threshold for Enhanced LLM Transparency), must be updated annually after initial submission and when new

LLM or other AI models are introduced in contract performance. Content section 3 (Noncompliant Output

Reporting) must be delivered annually by June 30.

Format: Unless otherwise directed, the data requested in this DRD must be delivered to the Government in an electronic format readable by a Government device utilizing the standards in NASA-STD-2804, “Minimum

Interoperability Software Suite.”

Interrelationship:

Applicable Documents:

Executive Order 14319, Preventing Woke AI in the Federal Government

OMB Memorandum M-26-04, Increasing Public Trust in Artificial Intelligence Through Unbiased AI Principles

Definitions:

LLM: Executive Order (E.O.) 14319, Preventing Woke AI in the Federal Government, defines LLM to mean “a large language model, which is generative AI model trained on vast, diverse datasets that enable the model to generate natural-language responses to user prompts.”

Unbiased AI Principles: OMB Memorandum M-26-04, Increasing Public Trust in Artificial Intelligence Through

Unbiased AI, identifies the following two Unbiased AI Principles:

(1) Truth-seeking:

LLMs must be truthful in responding to user prompts seeking factual information or analysis. LLMs must prioritize historical accuracy, scientific inquiry, and objectivity, and must acknowledge uncertainty where reliable information is incomplete or contradictory.

(2) Ideological Neutrality:

LLMs must be neutral, nonpartisan tools that do not manipulate responses in favor of ideological dogmas.

Developers must not intentionally encode partisan or ideological judgments into an LLM' s outputs unless those judgments are prompted by or otherwise readily accessible to the end user.

Scope: The contents of this DRD provide information to demonstrate LLM or other AI model compliance with the

Unbiased AI Principles.

Contents:

1) Minimum Threshold for LLM Transparency: The following information must be provided with initial submission and updated in accordance with the submission frequency defined in this DRD.

i. Acceptable Use Policy:

Provide the acceptable use policy from the original LLM developer. This acceptable use policy should characterize and differentiate appropriate and inappropriate use of the product offering.

ii. Model, System, and/or Data Cards:

Provide the model, system, and/or data cards which outline essential information about the model, system, and/or data as it relates to the product offering. Information provided should include summaries of the training process, identified risks and mitigations, and model evaluation scores on LLM benchmarks.

iii. End User Resources:

Provide product tutorials, developer guides, or other best tools to help customers ensure proper use of the

LLM and maximize utility.

iv. Mechanism for End User Feedback:

Describe the mechanism to be used by the Contractor to enable submission of feedback from the end-user to the Contractor relative to outputs of LLM, or other AI models, that are considered by the user to violate the

Unbiased AI Principles. Examples of such a mechanism could include a general email inbox, specific product point of contact, or similar mechanism for providing feedback from the end-user to the Contractor.

2) Threshold for Enhanced LLM Transparency: The following information ☐ is or ☐ is not required with initial submission of this DRD. Submission of the following information may be required to supplement the information required by the Minimum Threshold for LLM Transparency, as directed by the contracting officer.

i. Pre-Training and Post-Training Activities:

A. Describe actions undertaken that would impact the factuality and grounding of LLM outputs.

B. System-level prompts that provide natural language instructions to the model specifying guidelines on responding to user-generated queries, particularly with regard to how a model responds when reliable information is incomplete, contradictory, or subject to individual interpretation.

C. The type of outputs restricted via content moderation and safety filters.

D. Use of red teaming as a means of continuously assessing the model to protect against incidents of bias in generated output.

E. Any training or development that occurred outside the United States, to include the type of activity and the country in which it occurred.

F. Any modifications or configurations undertaken to comply with any regulation from a government other than the U.S. Federal Government.

ii. Model Evaluations

A. Results of bias evaluations conducted by the vendor and the methodology for performing such tests (e.g., testing prompt pairs for politically-oriented topics).

B. Benchmark scores for vendor evaluations to measure a model’s bias, helpfulness, honesty, or accuracy when provided with ambiguous versus straightforward questions. In some instances, benchmarks to compare performance across multiple languages will be necessary.

iii. Enterprise-Level Controls

A. Governance tools, such as customizable system instructions that are additive to a base model’s system prompts or content generation filters.

B. LLM model evaluation tools that enable comparisons either between a model’s outputs, or across various models.

C. Product features that require a model to cite sources of its outputs or otherwise provide visibility into the provenance of the model’s outputs.

iv. Third Party Modifications

Disclosure of additional controls to modify an LLMs output applied by the vendor if the vendor is not a direct developer of the LLM (e.g., classifiers, system prompts, fine-tuning, content moderation and filters).

3) Noncompliant Output Reporting:

The Contractor must deliver reports summarizing the Contractor’s tracking, disposition, and remediation of end user feedback regarding LLM or AI model outputs in violation of the Unbiased AI Principles. Reports must be delivered in the format specified by the contracting officer and in accordance with the submission frequency defined in this DRD.

Data Sensitivity. Submission in accordance with this DRD must not compel the disclosure of sensitive technical data

(e.g., model weights) beyond what is necessary to confirm alignment with the Unbiased AI Principles.

Remarks: None.

Maintenance: Updated submittals, as required by the Submission Frequency section of this DRD, are only required when there have been changes to the data addressed in the Content section of this DRD. Updated submittals must include an executive summary of changes. If nothing has changed, the Contractor must provide notification in writing to the contracting officer to document inapplicability of the requirement for an updated submittal.