030ADV23R0055 - Data Science Support QA Release Part 1 of 2.pdf

030ADV23R0055 - Data Science Support Questions Answers 1 of 2

QA# Solicitation Section Question Library Response

1 Section L.4.3 Past Performance

In section L.4.3 Past Performance, it states, “the offeror’s selected references shall return the completed questionaires directly to the contracting officer”. Please clarify if this is correctly stated, and if so, what is considered to be the “questionaire"? The requested description in L.4.3.(v) would require that our references have familiarity with the requirements in the Library’s RFP in order to provide relevant answers. We appreciate clarification on this requirement.

The offeror’s selected references shall return the completed questionnaires directly to the contracting officer, Delonte McKinney at dmckinney@loc.gov and the contract specialist, Julia Hanson at juhan@loc.gov by the due date for proposals." has been removed from the solicitation.

If the Alternate Program Manager is a key personnel, is it expected that they be included in developing the budget (e.g. they should have billable hours along with the Program Manager) or is their inclusion to be treated as a replacement for the named Program Manager for redundancy purposes and only their resume qualifications are evaluated?

Please see the revised key personnel listed in Section C and H.7 LC52.237-1 KEY PERSONNEL of the solicitation.

3 Section M.2.4

In section M.2.4 it states “The Library may determine that an offer is unacceptable if the option prices are signifcantly unabalanced.” This requirement would make sense when the nature of the work in the Option periods is similar/equivalent in nature to the work from the Base period. However the two Option periods here vary greatly in scope of work compared to each other and the base period. The base period is the development of the data science capabilities with an estimated total team of 10 personnel, while the first Option is hosting the developed data science capabilities, and the second Option period is transitioning the developed capabilities to run on internal LOC infrastructure. Could you clarify if the expectation is that the Option period prices are of similar size to the Base period?

"The Library may determine that an offer is unacceptable if the option prices are significantly unbalanced." has been removed from the solicitation.

4 Section 9.1

In section 9.1, “The Library will not separately reimburse the contractor travel expenses.” Is our understanding correct that travel should still be line-itemed as its own set of expenses in the budget justification or is the desire to have this included in the overhead for the final hourly rates? Any clarification is appreciated.

The contractor shall provide firm fixed pricing that includes travel to the Library campus. The work shall be primarily completed at the contractor's facility. Less than 5% of work is expected to be on-site.

5 Section 7.1

In section 7.1, could you clarify how many onsite visits and what durations/percentages of onsite work should be calculated into the budget?

The work shall be primarily completed at the contractor's facility. Less than 5% of work is expected to be on-site.

Is this expected only from certain Key Personnel such as the Program Manager and/or for certain events?

The person doing the training would have to report to the site.

Is there an incumbent on this contract?

No

Would the government please extend the proposal due date by 2 weeks to 31 August?

Proposals shall be due by 08/28/2023, 11 AM Eastern Daylight Time

9 Section L

Would government please exclude the cover pages, table of contents, appendices, and attachments from the 27-page limit?

See updated Section L.

10 Section L.4.2; Section 8.1

Section L.4.2 calls out 5 key personnel, and H.7 lists 5 positions as key. However, Section 8.1 only provides 4 position descriptions.

• Section 8.1 does not list the Alternate PM as H.7 does.

• Would the government please confirm that:

o The Alternate PM role can be filled by one of the other key personnel positions?

o That a resume is not required for the Alternate PM?

Please see the revised key personnel listed in Section C and H.7 LC52.237-1 KEY PERSONNEL of the solicitation.

• Would the government please provide a full list of the technologies now in use?

• This should include databases, analytics, and visualization tools.

For analysis, CRS uses MS Excel, MS Access, R/R Studio, Python/Jupyter on desktops. There is also limited use of a visualization tool called D3 and GIS staff use ARC GIS. SAS, STATA, and Python are available on a server accessed via a Windows Remote Desktop. Adobe Analytics for web traffic analysis of congress.gov. LIS, an office within CRS, uses Postgres databases, while we still also rely on the LIS Oracle databases. Congress.gov (CDG) uses SOLR as its search engine. Other languages/software in use on CDG include python, php, symphony, Django, Jenkins and RabbitMQ. We use Confluence, GitLab and JIRA for documentation and issue tracking. We use WordPress for creating help pages. Markup languages include XML, HTML and JSON.

12 Section C

For the Health Insurance Pricing Project

a. Has the Library of Congress compiled this data into a single public website, or is the vendor required to scrape data from Health Plan websites and compile into a single public website?

b. If LoC already has a public website, would they provide a link

This project will require scraping Health Plan websites. The ultimate deliverable will not be for a public website. Data will be compiled to be analyzed internally by CRS staff (with assistance from the contractor) to address questions of interest to Congress.

13 Section C

For Topic 3 (the Empower migration) -how large is the existing SAS database to be migrated? High level statistics will be helpful to evaluating the required effort (number of data tables, number of personnel records, size in GB, etc.)

This would be data for approximately 650 staff members. The records are downloaded from Empower Insight and then agency specific info, like CRS titles, contact info and other info needed for publishing are added.

14 Section 6

In Section 6. Contract Performance, LoC stipulates in “4.1 Data Models” [to be delivered] “Within 330 days of contract award”:

a. Would LoC provide guidance on the priority and sequence of the data model projects, as this is important input to staffing

b. Would LoC provide estimated timing to preferred delivery date of each data model

This is completely at the contractor's discretion.

Contractor should process the models in the way they see as most efficient.

15 Section C What cloud hosting service(s) does LoC currently utilize, and in what capacity?

The Library 's primary cloud hosting service is Amazon Web Services (AWS). We also use Azure for limited applications.

Who are the incumbent vendors working on these or similar projects within LoC?

There are no contractors currently providing this type of support to CRS.

Would LoC relax the US citizenship requirement to include Green Card holders, or H-1B employees with relevant US education and work experience; what about Green Card holders with US Citizenship applications pending, all subject to the same background checks?

This requirement remains unchanged.

Does LoC have a past Performance template we should use, or is vendor permitted to create their own within the required guidelines?

The offeror’s selected references shall return the completed questionnaires directly to the contracting officer, Delonte McKinney at dmckinney@loc.gov and the contract specialist, Julia Hanson at juhan@loc.gov by the due date for proposals." has been removed from the solicitation.

Does the FedRAMP requirement exclusively pertain to the hosting services for tools and datasets, or is it also applicable to the tools themselves? Additionally, is compliance with FedRAMP criteria deemed acceptable, or is it obligatory to attain FedRAMP certification?

It would be for the hosting. Please review Section C for FEDRAMP certification requirements.

Could the Library provide further clarification on "The contractor shall be FEDRAMP certified" as outlined in section 4.3 under HOSTING TOOLS?

Please review the information listed on the FEDRAMP website for further information https://www.fedramp.gov/

21 Section M Could you provide a breakdown of the weightage assigned to various evaluation factors?

Please see Section M.2. Evaluation Factors

Can the Library confirm whether secret or top-secret level security clearance is required for the key personnel?

No, there is no need for secret or top secret clearance.

23 Section C

In multiple sections of the SOW, the Library/CRS mentions existing models and tools. Excel was the only software mentioned. Would you please provide a summary of the tools (e.g., SAS, R, Python, etc) CRS analysts use and also those they have access to (if different from the ones they use).

For analysis, CRS uses MS Excel, MS Access, R/R Studio, Python/Jupyter on desktops. There is also limited use of a visualization tool called D3 and GIS staff use ARC GIS. SAS, STATA, and Python are available on a server accessed via a Windows Remote Desktop. Adobe Analytics for web traffic analysis of congress.gov. LIS, an office within CRS, uses Postgres databases, while we still also rely on the LIS Oracle databases. Congress.gov (CDG) uses SOLR as its search engine. Other languages/software in use on CDG include python, php, symphony, Django, Jenkins and RabbitMQ. We use Confluence, GitLab and JIRA for documentation and issue tracking. We use WordPress for creating help pages. Markup languages include XML, HTML and JSON.

We have noted a number of federal agencies that have intentionally migrated away from SAS and other legacy programs to R and Python. Is there a directive or other driver for CRS to make similar moves?

There is no directive to make this move, but CRS has been exploring R more in the last few years.

How is CRS currently operating?

What tools are they using for analysis?

Research Divisions use SAS, R/R Studio, Python/Jupyter, Excel, and Stata. Support and operations use SQL for congressional data ETL and Adobe Analytics for web metrics.

In what ways do you think AI and Data Science will improve the current CRS operations?

The research divisions of CRS are looking to Data Science for advancing and expanding our leagle and policy research, not necessarily to improve “operations.” Expanding our expertise in research-oriented DS, however, will likely spillover to improved operations.

Do we have any applications in place that CRS is currently using, and the application should be enhanced with AI? Please provide more information

The CRS applications that could be enhanced using AI include: (1) Assigning terms to bills, (2) Creating drafts of bill summaries, and (3) Managing the workflow of bill summaries. CRS also uses a Text Analysis Program (TAP) that could be augmented by

AI.

What is your definition of models?

Are you talking about Machine Learning Algorithms for prediction, classification, clustering, etc.?

The contractor is free to suggest any techniques that would be best for each model.

Do we have any domain expertise and Data expertise in place to assist us in clearing any doubts we may have about the problem statement, objectives, or Data?

In most cases, CRS analysts will be familiar with the data sets and would be available to assist the contractors. CRS has data expertise on its staff, though there are technical and other impediments to fully engaging that data expertise.

Can we have direct contact with stakeholders to make sure we stay in line with requirements?

Yes, there will be direct contact with stakeholders.

Can you please provide more information on the vision of the tools that need to be developed that require Confidentiality – Objectivity - Non-partisanship – Authoritativeness

– Timeliness? Are you looking for applications with AI capabilities?

Machine Learning testing models, BI dashboards?

Contractor is free to suggest any type of tools they feel will best meet the model needs. The contractor shall ensure the tool remains neutral in its interpretation of data.

If we assume you need an application that runs on a desktop or cloud?

What is your vision of that? What functionalities will you need besides AI capabilities? In what manner will the end user be utilizing it? Does it require password protection?

Data will be public data. However, some specific scenarios run may be confidential. So, the details of scenarios run should be unaccessible. Staff should be able to run the tools from their laptop computers. Contractor is free to suggest capabilities they feel can best create the models.

What do you mean by Big Data?

Large Data or Live stream Data, real-time data, etc.?

Some of the data sets are very large and may have frequent updates.

What is your vision for quantitative tools and modeling? Are you looking for smart applications with AI capabilities that take any data and produce an output through analytics?

We are looking for data analysis tools that produce insights that traditional quantitative tools could not.

Topic 3 is an application for internal use. The contractor shall build a new, cloud-based database that could return queries in a number of formats (e.g., excel .csv and data visualization)

Assuming you need a smart application, how many tools are you looking for?

We are looking for a tool for each of the models described. If the same tool can be used for multiple models, that is fine.

Assuming you need Machine Learning models to model Data, how many models are you looking for?

We are looking for a tool for each of the models described. If the same tool can be used for multiple models, that is fine.

What is the current CRS work model, products, and data requirements?

CRS products are widely available online and consist primarily of reports and analysis. Please see https://crsreports.congress.gov/ for further information.

Can you please provide more information on monitoring current work outside of CRS?

Please see Section C.6 Contract Performance Schedule of Events and Deliverables for further information.

Is it required to respond to Hosting Services?

Yes

40 Section L.4.3

Regarding section L4.3 – Total Dollar amount – Is that the total Contract Value or Total Amount that has been received?

Total contract value amount

What is the final deliverable? A prototype or a fully functional deployed system?

Final deliverable is fully functioning tools and training on those tools.

What are the key metrics to judge that the model and data analysis is good?

The tool needs to produce usable data that is verifiable to ensure our authoritativeness mission requirement is satisfied.

Is all the data already available and easily accessible?

Yes.

44 Section 8.1, Pg 14

Please confirm the Library’s intent is to engage each 3-person team as full time efforts once initiated as opposed to intermittent efforts throughout the PoP.

Please see the revised Section C.8 Special Qualifications for further information

45 Section B.1, Pg 4

Please confirm the PoP for the below is a total of 10 months, starting with the effective date of option award and extending to 10 months post award.

“Period of Performance: 10-Months After Effective Award Date of the Option”

The period of performance shall be 10-Months After Effective Exercised Date of the Option.

46 Section B.1, Pg 5

Please confirm the PoP for the below is a total of 6 months, starting with the effective date of option award and extending to 6 months post award.

“Period of Performance: 6-Months After Effective Award Date of the Option”

The period of performance shall be 6-Months After Effective Exercised Date of the Option.

47 Section 8.1, Pg 14

Given there is no information on when the additional teams will/may be required by the Library, how are offerors to price the number of hours for the price buildup in the pricing volume?

“It is estimated that eventually three separate project teams of three individuals each (total of nine personnel) may be needed for the modeling phase.”

Please see the revised Section C.8 Special Qualifications for further information

48 Section 8.1, Pg 14

Please include the optional CLINS in Section B for the additional teams that may eventually be needed.

“It is estimated that eventually three separate project teams of three individuals each (total of nine personnel) may be needed for the modeling phase.”

Please see the revised Section C.8 Special Qualifications for further information

49 General Are the requirements for Alternate PM the same as the PM?

Please see the updated Alternate Program Manager description in Section C

50 General Is any data anticipated to be “Security Category: Moderate” or “Security Category: High”?

Data is public information. The contractor will not have access to confidential information

51 Section 7.2

Can the government confirm that within the computing environment, the contractor will have access to all relevant software, such as Python, Tableau, R, JavaScript?

7.2. “Government furnished property includes workstations, laptops, access to existing software and development environment.”

The contractor will have access to the relevant software needed to fulfill the requirement.

Page 1, Solicitation Cover page, Block 8.

Would the Government please consider extending the proposal deadline one week from 11am on Friday, August 18 to 11am on Friday, August 25?

Proposals shall be due by 08/28/2023, 11 AM Eastern Daylight Time

Page 4, Section B.1 Pricing/Cost Schedule. The Government states that Periods of Performance (POP) for the "Optional Model Hosting IAW SOW Section 4.3" will be "10- Months After Effective Award Date of the Option," and for the "Optional Transition from External to Internal Hosting IAW SOW Section 5" will be "6 Months After Effective Award Date of the Option." The POP of the other tasks is Sept 29, 2023 to Sept 28, 2024.

If exercised, would the Government anticipate awarding the two optional tasks during the Base Year's 12-month period? Or might these optional tasks extend beyond the anticipated Base Year POP? At what point would the Government anticipate determining whether or not to exercise these optional tasks?

The period of performance shall be for the times listed in each CLIN, after exercised. The government will exercise the options if/when needed.

Page 6-7, Section 1, Overview/Background, Objective 3: Maintain existing services

Are the five topical divisions and one support group members of the CRS Data Steering Committee?

The steering committee includes personnel from across CRS.

Page 14, 8.1 Key Personnel | "It is estimated that eventually three separate project teams of three individuals each may be needed for the modeling phase.

It is also anticipated that CRS will require one lead project manager to support the requirements herein (10 total)."

Does the Government have an anticipated level of effort for each of the five key personnel positions (Program Manager, Alternate Program Manager, Software Engineer, Digital Interface/Data Visualization Specialist, Data Scientist) throughout the duration of the Period of Performance? How does the Government anticipate these Key Personnel would be aligned to the three separate project teams during the modeling phase?

Can the Offerer propose alternate project team compositions?

Please see the revised Section C.8 Special Qualifications for further information

Page 58, L4, Proposal Content.

"The Technical Approach and Staffing proposal sections are limited to 27 pages to include all documents. This is to include cover pages, table of contents, appendices, and attachments."

Does the 27 page count include the three Past Performances? Does each Past Performance have a page limit (e.g. 2 pages)? Where in the proposal should bidders include the Past Performances - e.g.: the Appendix?

See updated Section L.

Page 60, M.2. Evaluation Factors | "The following evaluation factors apply to this acquisition, listed in descending order of importance:

• Factor 1 Technical Approach

• Factor 2 Staffing

• Factor 3 Past Performance

• Factor 4 Price Technical Approach is more important than Staffing and Staffing is more important that Past Performance. Technical Approach, Staffing, and Past Performance, when combined, are approximately equal in importance to Price."

As currently written, the evaluation factors appear to contradict themselves. One on hand, the evaluation factors as listed would appear that Technical Approach is the highest evaluated factor with Price being the lowest evaluated factor. However, the line "Technical Approach, Staffing, and Past Performance, when combined, are approximately equal in importance to Price ." This would appear to make Price weighted as 50% of the evaluation factors, with Technical Approach, Staffing, and Past Performance to account for the other 50% combined. This would suggest this procurement is Lowest Price Technically Acceptable (LPTA). Could the Government please deconflict these evaluation factor statements?

Technical Approach is more important than Staffing and Staffing is more important that Past Performance. Technical Approach, Staffing, and Past Performance, when combined, are approximately equal in importance to Price.

Page 4, Section B.1 Price/Cost Schedule and the Attachment J1 Data Science Models Pricing Sheet

The solicitation's cost structure is currently planned to be Firm Fixed Price with an itemized cost for each major requirement. Firm Fixed Price cost schedule typically fit projects with specific scope and technical requirements, which allow for more accurate assessment of the needed level of effort. However, the extent of major technical data requirements of this solicitation are difficult to assess for level of effort, such as the analysis of CRS current capabilities and tools (Sec. 3.4), reporting on activities of other organizations in the field of data science (Sec. 3.7), and the creation of data modeling tools (4.1). Will the Government consider utilizing a Time and Materials cost structure to ensure effort and funding were allocated in a way to maximize value during the period of performance?

This is a Firm Fixed Price solicitation.

Page 8, Section 3.4 ANALYSIS OF

CRS'S CURRENT CAPABILITIES

AND TOOLS

The contractor shall examine and determine the current capability within CRS. This would include studying the CRS work model, products and data requirements in many topic areas. As part of the analysis, the contractor shall identify ways CRS can better utilize its existing tools.

This requirement description mentions "many topic areas" and "existing tools" without providing much content on the scope of these.

Given the Government's current intention of utilizing a Firm Fixed Price cost schedule, it's important to be able to assess scope and level of effort. Will the Government consider providing more clarity on the extent of this requirement, such as the number of topic areas or the number and nature of existing tools?

Please see the topic areas discussed in Section 4.1 CREATION OF DATA MODELING TOOLS for further information.

Page 9, Section 3.7 REPORTING

ON EXTERNAL DATA ADVANCES

The contractor shall monitor current work outside of CRS on quantitative analysis related to policy issues, and using applicable new methods, including machine learning/artificial intelligence.

This shall include periodically scanning analytic products and analysis from policy research organizations and other public and or private entities.

Given the Government's current intention of utilizing a Firm Fixed Price cost schedule, it's important to be able to assess scope and level of effort. Will the Government consider providing more clarity on what frequency "periodically" is intended to have and how extensive the report on findings is supposed to be?

The report is to be given twice a year.

Pages 9-10, Section 4.1

CREATION OF DATA MODELING

TOOLS

The contractor shall use the topics below to identify, develop, and put into production models and/or tools of varying levels of complexity... Tools shall be functional in a production environment by within the period of performance of the contract. Deployment of models may be staggered, but all models shall be developed within 9 months of contract award to allow for staff to work with contractor to fully understand the tool.

Q1. Given the Government's current intention of utilizing a Firm Fixed Price cost schedule, it's important to be able to assess scope and level of effort. Does "use the topics below" mean that the five topics/projects listed on Page 10 are the exact ones that the contractor will be required to deliver, or are they just examples of potential projects? If the former, is it an exhaustive list or could other projects be added during the period of performance?

Q2. Given the expedited timeline of 9 months for putting all functional tools into a Production environment while providing the other requirements along the way (e.g., training, forums, research), will "functional" be defined and agreed upon at the initiation of each tool development to avoid scope creep or enhancements that could delay deployment into Production?

The models are the ones that are to be built. No models will be added. The definition of a "completed, functional" tool will be established with the initiation of the project.

62 Page 58

The solicitation indicates “Offerors must submit the following minimum information for the identified key personnel in resume format”. To ensure our response meets this requirement while still being mindful of the page limit, could LOC clarify what “resume format” is expected to be?

Format should list past experience and skills.

63 Page 10

The RFP suggests the existence of a student loan repayment calculator. Is there any information that can be provided in regard to this existing calculator, such as, input/output, control variables, underlying models, etc.?

The student loan repayment calculator is programmed in Stata. It is used to model estimated student loan repayment outcomes for a variety of hypothetical borrower cases, based on repayment according to selected income-driven repayment (IDR) plans. Borrower cases are designed based on a number of factors such as degree type, student loan amounts, and income during repayment using data from the National Center of Education Statistics National Postsecondary Student Aid Study and the American Community Survey. Repayment outcomes include: the number of payments after which the loan is retired; monthly payment amounts over the course of repayment; the total amount paid by the borrower in principal and interest; and when applicable, the total amount subsidized or forgiven by the federal government, in principal and interest.

The model takes into account a number of assumptions regarding inflationary adjustments to income and poverty thresholds, student loan interest rates, borrower circumstances, and other underlying principles.

I.9 LC52.232-25 PROMPT

PAYMENT (DEV)

States “The Library of Congress is not an agency subject to the Prompt Payment Act. 31 U.S.C. 3901(a)(1)”.

Are you able to provide a Schedule of Payments?

Payment shall be in accordance with Section B.

Could you please provide additional clarification regarding I.3 52.217-8

OPTION TO EXTEND SERVICES (NOV

1999)?

Please see FAR 52.217-8 for further information.

L.1 LC52.233-2 PROTESTS TO

THE LIBRARY OF CONGRESS (DEC

2013), section L.4.3 Past Performance

States “References may be for the prime or subcontractors; however, at least one (1) reference shall concern the relevant past performance of the prime.” Is it necessary for the Offeror to have a history of prime contracting?

References may be for the prime or subcontractors;

however, at least one (1) reference shall concern the relevant past performance of the prime. The information must be clear as to whether the work was done by the offeror as a prime contractor or as a subcontractor, or by a subcontractor working for the offeror.

Could you please provide clarification regarding SECTION C -

DESCRIPTION/SPECIFICATIONS/STAT

EMENT OF WORK, section 4.

CREATION OF DATA MODELING

TOOLS. Specifically relating to the necessity of model creation and/or tool creation. It appears that the topics primarily require data integration and reporting via business intelligence.

The contractor is free to suggest any techniques that would be best for each model. The government shall be able to manipulate the data effectively.

How would the government know the offeror's Unique Entity Identifier number if there is no requirement to provide?

"Offerors shall include their Unique Entity Identified (UEI) number in their technical volume information." has been included in Section L.4.1

69 Section C

Would the Library of Congress (LoC) point vendors to publicly available sources that provide examples of the in-scope data sets, including:

a. DOD Budget justification PDFs and spreadsheets

b. Historic Justification for Dept. of State and Foreign Operations spreadsheets and other appropriation documents

Yes, all publicly available. Please see https://comptroller.defense.gov/Portals/45/Docum ents/defbudget/FY2024/FY24_Green_Book.pdf, https://comptroller.defense.gov/Portals/45/Docum ents/defbudget/FY2024/FY2024_m1.pdf, https://www.state.gov/plans-performance-budget/international-affairs-budgets/ https://www.state.gov/fy-2024-international-affairs-budget/

Could you please provide additional clarification regarding H.10 LC52.239-

2 SYSTEMS DEVELOPMENT LIFE

CYCLE (SEP 2018)? Specifically related to section (a) “Requirements:

The Contractor shall gather and analyze requirements and produce a system requirements document

(SRD)”.

Please see H.10 LC52.239-2 SYSTEMS DEVELOPMENT LIFE CYCLE (SEP 2018) If the contractor's proposal includes the development and deployment of an application to be used in the Library's production environment, this clause shall apply.

Questions & Answers