Final RFP National Transit Database (NTD) - Program Support RFP.pdf

National Transit Database (NTD) Program Support Request for Proposal No. 69319522R000002

SECTION B: SUPPLIES OR SERVICES AND PRICES

B.1 Supplies or Services The United States (US) Department of Transportation (DOT), Federal Transit Administration (FTA) anticipates awarding a single Small Business Set-Aside, Firm-Fixed Price (FFP) and Labor-Hour hybrid contract. This requirement is a total small business set-aside. The Period of Performance (POP) will be a three year and one-month term, consisting of one thirteen-month base period and two 12-month (1-year) option periods.

The FTA has an existing National Transit Database (NTD) program support contract. This acquisition seeks to find a firm to continue these services in accordance with the contract line items (CLINs) of this Request for Proposal (RFP).

The North American Industry Classification System (NAICS) Code for this acquisition is 541611, Administrative Management and General Management Consulting Services. The small business size standard is $16.5 Million.

B.2 Contract Line Item Number (CLIN)

Base Period (13-Months) CLIN 0001 - NTD Assistance (Firm-Fixed Price) CLIN 0002 - Annual Module (Firm-Fixed Price) CLIN 0003 - Monthly Module (Firm-Fixed Price) CLIN 0004 - Safety and Security Module (Firm-Fixed Price) CLIN 0005 - Apportionment and certain non-excel data products (Firm-Fixed Price) CLIN 0006 - Annual Review (Firm-Fixed Price) CLIN 0007 - Reporting Manuals, Training Materials and Website Content (Firm-Fixed Price) CLIN 0008 - Program Performance Metrics Reporting (Firm-Fixed Price) CLIN 0009 - Travel (Labor Hour) CLIN 0010 - Special Projects (Labor Hours) CLIN 0011 - Training Provision (Labor Hours) CLIN 0012 - User Acceptance Testing and Data Modeling (Labor Hours) CLIN 0013 - Transition Support (Firm-Fixed Price) CLIN 0014 - Written Documents (Labor Hours) CLIN 0015 – Work Plan (Firm-Fixed Price)

*NOTE: For each respective year after the base period, the CLIN number will change respectively yet maintain the same title etc. For example, in the first option year, CLIN 0001 will become CLIN 0101.

B.3 Contract Period of Performance The contract period of performance is from date of award through a three year and one-month period, consisting of a 13-month base and two (2) one-year option periods. The Federal Acquisition Regulation (FAR) clause 52.217-8 “Option to Extend Services” NOV 1999, shall be incorporated into the resulting contracts, as referenced in Section I, in the event the Government requires continued performance of any services with the limits and at the rates specified in the contract beyond the period of performance term.

B.4 Pricing/Labor Rates Table

Use Attachment J-1 Pricing Matrix to input pricing breakout.

*NOTE: The labor categories in the Pricing Template (Attachment J-1) shall be used for the total life of this contract. If the labor categories for this requirement change but within scope of the contract, a modification must be executed to clearly reflect this to ensure clarity concerning the labor categories authorized for use. This contract will be awarded to the contractor based within the pre-established labor rates as stated using the CLIN structure in the Pricing Template (Attachment J-1).

(END OF SECTION B)

SECTION C: STATEMENT OF WORK

C.1: Statement of Work

Background

FTA’s National Transit Database (NTD) was created to help meet the needs of individual public transportation systems, all levels of government, and the public for information on which to base public transportation service planning. It is the Nation’s primary repository of public transportation data. The NTD collects financial data, operating statistics, safety events, asset conditions, and more from American transit systems. FTA uses NTD data to apportion funding to urbanized and rural areas in the United States, to inform decision makers at all levels of government, and for research purposes.

FTA has an existing NTD program support contract furnishing services described herein. This acquisition seeks to find a firm to continue these support services in accordance with the Statement of Work described herein.

C.2 Tasks

NTD Program NTD support services consist of three primary functions:

• Data Collection

• Validation

• Data Products

Data Collection The NTD program collects transit data. The NTD program is a longitudinal survey of the transit industry—it collects data from the same organizations year after year. However, each agency may have different individual system users who physically report these data in the online system.

This makes data collection and validation challenging since these users change often, and the NTD program is required by law to collect data according to uniform system of accounts.

Therefore, the NTD program provides significant assistance and training to its client users each year to ensure that data is reported to the NTD program consistently according to the specific standards and definitions established by the program. Additionally, the NTD program does extensive follow-up to remind its client users to complete their reports on-time, or to complete past due reports.

Validation The NTD program validates the submitted data. Validation is a critical program component to ensure that the data meets the standards and definitions described in the NTD Policy Manual.

Validation ensures that NTD data used by decision makers at all levels of government is reliable.

Validation also ensures the proper distribution of billions of dollars of Federal funding that rely on NTD data each year. Data is validated at using two different processes: (1) in real time when a user initially enters the data on the NTD website; and (2) using NTD’s Branch Environment database (described further below).

http://www.dot.gov/ost/m60/tamtar/tar

Data Products The NTD program uses validated data to produce apportionment and other data products. CLIN 0005 lists the relevant data products for this procurement. In total, there are approximately 15 apportionment data products and two non-Excel data products produced on an annual basis.1 The apportionment data products determine the distribution of billions of dollars in formula funding, according to Federal law. There are also approximately 20 special analysis data products produced at various times during the year under CLIN 0010.

General Information

1. NTD Data: From Data Collection to Data Product

The NTD production process begins when individual transit agencies begin assembling internal records on their transit operations according to FTA’s uniform system of accounts and the NTD Policy Manual. The NTD Program provides assistance and training to transit agency staff to support this process. Transit agency staff then input data into the NTD reporting website.2 This website uses the Appian cloud computing enterprise software application. The website prompts a transit agency user to complete and submit NTD forms. FTA publishes a user guide that describes the physical mechanisms how users can enter their data.3 Once submitted, FTA validates the data before publishing the data in different data products.

2. Information Technology (IT) and Data-Related Processes

The NTD program primarily relies on two databases: (1) Appian database (‘Appian’), and (2) Branch Environment database (‘Branch’).4

Appian Database The Appian Cloud is a platform-as-a-service (PaaS) solution that collects and houses NTD data.

Appian houses NTD’s transactional database for data collection and serves as the NTD system of record. Appian serves as the ‘front end’ to the NTD. Each night, FTA moves the collected data through a pipeline of extract, transform and load processes. These processes create copies of Appian database data in the Branch Environment database.

Branch Environment Database The Branch Environment (‘Branch’) is an on-premise database housed at the Department of Transportation (DOT). Its purpose is to support data validation activities and production of data products. It is a separate database from ‘Appian.’ Information Technology security (IT security) is managed by FTA’s Office of Administration (TAD) and the contractor is not required to manage Branch’s IT Security.

1 NTD data products listed on the following NTD Data webpage are not covered in this RFP:

https://www.transit.dot.gov/ntd/ntd-data 2 https://www.transit.dot.gov/ntd/national-transit-reporting-tool 3 https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf 4 There are multiple databases in Appian. However, these related Appian servers contain largely the same data.

Thus, this RFP refers to the Appian database as a single database.

https://www.transit.dot.gov/ntd/ntd-data https://www.transit.dot.gov/ntd/national-transit-reporting-tool https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf

Branch houses a copy of the Appian database. Branch also houses other industry-standard database objects, many derived from the copy of the Appian database (e.g., indexes). FTA’s TAD Office owns and is responsible for making Branch accessible. The copy of the Appian database is limited to ‘select’ privileges. The contractor will have permission to manage the other aspects of Branch to perform data validation and production of static data products as needed. These permissions include the following:

• Creating, editing, and deleting database objects

• Creating, editing, deleting, and executing events, views, queries, stored procedures, and tables

• Creating, editing, deleting, and executing queries

All data, database objects, such as events, views, queries, stored procedures, tables, and related documentation are owned by FTA. Branch is currently a MySQL database, but FTA may transition to a different database type in the next year. It is expected that the transition will be to MariaDB, which is very similar to MySQL. Regardless of the new database type, TAD will assist with this transition in terms of data migration and meeting TBP’s business requirements.

NTD Branch has approximately 50 million records spread across four transactional tables and approximately 300 referential tables.

IT Roles and Responsibilities Two FTA offices have support-related roles and responsibilities. First, TAD is primarily responsible for the day-to-day functionality of the Appian Database, the Appian online reporting interface, and the Branch system. TAD is responsible for the NTD front-end data portal where transit agencies enter their data, known as the Transit Integrated Appian Development (TrIAD) platform. Within the NTD data entry portal, TAD is responsible for all development and enhancements to the systems, and ensuring that they are live, active, and available. The contractor is responsible for ensuring that the captured data is accurate.

The second FTA office is the Office of Budget and Policy (TBP). With respect to NTD services, TBP’s role is that of a client. TBP staff and its contractor oversee the programmatic aspects of the NTD, including all business requirements, data collection, validation, and data products. TBP staff and its contractors prioritize bug fixes and system enhancements, and then also perform user acceptance testing before deployment. This also includes disaster recovery testing involving the NTD Appian database. TAD and TBP hold re-occurring coordination meetings to set these priorities and discuss emerging issues.

Many issues involve shared responsibility between TAD, TBP, and their contractors. For instance, if a user encounters an error, and the contractor detects the error, the contractor is responsible for creating a TAD support ticket. TAD is responsible for identifying a root cause and fixing the issue. The contractor helps TAD prioritize all the detected errors for resolution.

TBP and the contractor interact with the Branch database, and TAD provides software and support to connect to Branch at no cost to the contractor. TBP and the contractor execute all queries on the Branch Environment. TBP and TAD also provide access to Branch so that the contractor can access draft data that supports validation, reports, and other functionality.

Additional documentation about the roles and responsibilities of TAD, TBP, and its contractors is available upon request. The contractor is also provided read-only access to the front-end user interface for purposes of being able to see the end-user experience in the system. The front-end user interface does not support any direct queries or scripting activities.

Appian Report Packages – From Original Submission to Closeout The contractor manages data submitted to the NTD through the Appian web interface and ‘report packages.’ Report packages are a combination of different data points and different forms for a given module (see CLIN 0002 below for the Annual Module). These data move through stages of validation before being closed out. For instance, the earliest stage of review is the original submission, and the last stage is ‘closeout.’ If the data is perfect and there are no validation issues, then the data proceed quickly from original submission to closeout.

In most cases, submitted data requires revisions. FTA requires the incumbent to manage all aspects of processing the data. For instance, an agency might submit the data October 31st, but validation checks have flagged 50 data points requiring validation. Over the next several weeks, the contractor performs ‘validation’ with the NTD reporter. Validation refers to the staff follow-up and tracking to closeout all 50 issues. Each issue is resolved by the transit agency either revising the previously submitted data or else by providing a sufficient explanation for the data to the satisfaction of the validation analyst. The contractor uses the Appian web interface to assign certain issues or tasks for the submitting agency to resolve.5 The contractor is responsible for communicating with the agency and to ensure that the agency resolves the issues. When all 50 issues have been resolved, then FTA reviews and approves the final submitted package.

General Data Collection: Reporter Type

Each organization that reports to the NTD is assigned an NTDID. Each NTDID is assigned a reporter type. Different ‘reporter types’ submit different amounts of data. For example, full reporters fill in the most forms and fields, while reduced reporters fill out fewer forms and fields.

FTA uses reporter type to capture relevant levels of detail applicable to different types of operations. There are ten different reporter types which are as follows:

• Full Reporter

• Reduced Reporter/Tribal

• Separate Service Reporter

• Build Reporter

• Planning Reporter

• State Reporter

• Rural Reporter

• Reduced Asset Reporter

5 The Appian process, including examples of tasks and issues are listed in the NTD User Guide:

https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf.

• Group Plan Sponsor

In total, approximately 3,000 transit agencies report to the NTD.6 Detailed descriptions of each reporter type are described in NTD’s policy manual.7

i. Full Reporters A full reporter is a transit agency that serves one or more urbanized areas, and either is ineligible to be a reduced reporter or else does not request to be classified as a reduced reporter. In total, there are approximately 500 Full Reporters.

ii. Reduced Reporters A reduced reporter is a transit agency that serves one or more urbanized areas, is eligible to be a reduced reporter, and requests to do so. To be eligible for reduced reporting, a transit agency must have the following:

o Operate 30 vehicles or less across all modes and types of service;

o Not operate a fixed guideway mode; and o Not operate bus service on an HOV (high occupancy vehicle) lane.

These NTD reporters are typically small transit operators, but unlike rural reporters, they offer at least some service in a Census Bureau Urbanized Area (UZA). There are approximately 450 reduced reporters.

iii. Separate Service Reporters A separate service reporter receives or benefits from 49 U.S.C. §5307 urbanized area formula program funding, does not directly operate service, and instead provides funding for modes that are reported by another transit agency. As recipients or beneficiaries of §5307 funding they are required to report to the NTD, but since they do not operate transit service, they only complete a minimal NTD report. There are approximately 10 separate service reporters.

iv. Build Reporter A build reporter receives or benefits from §5307 funding, does not directly operate or contract out service, and is building or rehabilitating transit infrastructure. One difference between build reporters and full urban reporters is that no service is consumed or offered by the organization reporting. However, financial information is still required to be reported to the same as for full reporters. There are approximately 10 build reporters.

v. Planning Reporter A planning reporter receives or benefits from §5307 funding, does not directly operate or contract out service, and spends §5307 funding on planning activities. One difference between planning reporters and build reporters is that a service is being planned but is not yet being built.

There are approximately 10 planning reporters.

7 The policy manual is found here: https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf. Note, some reporter types are defined using technical transit terms, such as ‘fixed guideway’ or ‘high intensity busway.’ These terms are defined in the NTD Annual Policy Guide.

https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf

vi. State Reporters

State reporters are those states (including Puerto Rico, Guam, the Northern Marianas, and American Samoa) that receive 49 U.S.C. §5311 rural area formula program funding. They provide FTA funding to rural transit operators as subrecipients within their state and are required to report to the NTD on behalf of those subrecipients that are not already full reporters or reduced reporters. There are 54 state reporters.

Reporting forms for rural subrecipients are very similar to those for reduced reporters. The NTD system refers to these subrecipients as rural reporters.8 Because the state reporters report their data on their behalf, rural reporters are not listed separately in this section. There are approximately 1,250 rural reporters.

vii. Tribal Reporters Tribal reporters operate public transportation in a tribal area and receive or benefit from §5311 funding under FTA’s Tribal Transit Program. Reporting forms for tribal reporters are the same as those of reduced reporters (described above). There are approximately 150 tribal reporters.

viii. Reduced Asset Reporters Reduced asset reporters receive or benefit from FTA funding other than §5307 or §5311 funding, and own, manage, or operate capital assets used in providing public transportation services. The NTD collects asset data from these agencies, but not service data. There are approximately 600 reduced asset reporters.

ix. Group Plan Sponsor Group plan sponsors sponsor a transit asset management (TAM) group plan and report on behalf of transit agencies that would otherwise be reduced asset reporters. Group Plan sponsors function like states in that they report on behalf of other operators. The forms for group plan members are the same as those for reduced asset reporters. There are approximately 20 group plan sponsors.

3. General Data Collection: Forms

FTA collects NTD data according to a series of forms’ The NTD program uses one or two letters to identify each of the eleven categories of forms:9

• P Forms – Profile Forms

8 Reporting forms for rural subrecipients are very similar to those for reduced reporters. For instance, they fill out a single RR-20 form. Rural reporters also do not operate service within a UZA; however, their service may connect a rural area to a UZA. In these cases, rural reporters are required to attribute their service as serving the rural area, rather than the urban area. To attribute their service as serving a rural area, they must become either a full reporter or a reduced reporter. There are approximately 1,250 rural reporters, but the state reporters generally act on their behalf to enter the data into the NTD system. Thus, these 1,250 rural reporters have individualized data but very few interact with the Appian NTD data collection website as the state reporters are responsible for entering data on their behalf.

9 Screenshots of samples of reporting forms can be found here: https://www.transit.dot.gov/ntd/ntd-reporting-system-forms and here https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf.

https://www.transit.dot.gov/ntd/ntd-reporting-system-forms https://www.transit.dot.gov/ntd/ntd-reporting-system-forms https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf

• B Forms – Basic Forms

• F Forms – Financial Forms

• A Forms – Asset Forms

• S Forms – Service Forms

• R Forms – Resource Forms

• RR Forms – Reduced Reporting Forms

• FFA Forms – Federal Funding Allocation Forms

• D Forms – Declaration Forms

• MR Forms – Monthly Reporting Forms

• S&S Forms – Safety & Security Forms

Certain forms are only filled out once. For example, the P Forms and some of the B Forms are specific to a single NTDID. However, most forms, must be filled out for each mode and type of service (TOS) operated by the NTDID.

Mode refers to the characteristics of a unit of transit service operation. The NTD has established 20 modes, including heavy rail, commuter rail, light rail, streetcar rail, ferryboat, motorbus, commuter bus, bus rapid transit, demand response, and more.

Type of service refers to the organizational arrangement of the mode. The NTD has established 4 types of service, the most common of which are directly operated and purchased transportation (contracted out operations).

During the period of this contract, FTA intends to introduce a separate reporting by geographic service line. This will apply in approximately five cases where a single transit agency operates two geographically separated services with the same mode and TOS. Once introduced, a transit agency will have to report separate forms for each geographic service line, just as if they were separate modes or separate TOS.

Forms also vary in complexity as some require users to complete more data fields than others.

The average NTD form has 56 fields, with a range of 2 to 227 fields on the shortest and longest forms, respectively.10 The following is a brief narrative summary of each form category:

i. Profile Forms (P Forms) The purpose of the NTD’s profile forms is to ask for basic information to complete the reporter profile. Collected data include agency name, doing business as name (DBA), address, etc. One form is used to identify the individual NTD system users for the agency and their roles. One form asks for the different modes and system profile information that will determine how many other forms will need to be completed. In total, there are four P-forms (P-10, P-20, P-30, and P- 40). The information on these forms changes infrequently.

10 For more differences between different forms within the same category of form, see FTA’s Annual Reporting User Guide. https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-09/2020%20NTD%20Annual%20Reporting%20User%20Guide.pdf

ii. Basic Forms (B-Forms) The purpose of the NTD’s two basic forms (B-10 and B-30) serve to collect information related to the service the transit agency provides, including basic organizational and service area information. For instance, FTA collects general information such as service area overlap with UZAs. Additional information includes contractual relationships and separate assets. However, a transit agency covering multiple UZAs or with multiple contractual relationships would have substantially more fields to complete.

iii. Financial Forms (F Forms) The purpose of the NTD’s financial forms is to capture financial data in a uniform manner using the NTD’s accounting system—the Uniform System of Accounts (USOA). The USOA specifies the manner in which transit agencies report financial and operating information reporting to the NTD by the transit industry, with the goal of collecting the full cost of providing public transportation.11 In total, there are five financial forms (F-10, F-20, F-30, F-40, and F-60).

The F-10 form (source of funds), F-40 form (adjustments to operating expenses), and F-60 form (balance sheet) are filled out once by each NTDID. The F-20 form (capital expenses) and F-30 form (operating expenses) are filled out once for each mode and TOS.

iv. Asset Forms (A Forms) There are six asset forms (A-10, A-15, A-20, A-30, A-35, and A-90). The purpose of these forms is to collect data related to transit assets to better understand the Nation’s asset inventory and their conditions. The information collected include transit asset management (TAM) performance measure targets, facilities inventory, revenue vehicle inventory, service vehicle inventory.

v. Service Forms (S Forms) There is only one service form (S-10). The purpose of this form is to collect service supplied and service consumed. The form is completed for each mode and TOS.

vi. Resource Forms (R Forms) There are two resource forms (R-10 and R-20). The purpose of these forms is to collect how many employees the transit agency employs and the number of maintenance-related service breakdowns.

vii. Reduced Reporting Forms (RR Forms) There is only one reduced reporting form. The purpose of this form is to collect consolidated data from small transit agencies covering, financial, service, safety events, and asset conditions.

viii. Federal Funding Allocation Forms (FFA Forms) There is only one Federal Funding Allocation form (the FFA-10). The purpose of this form is to allow the transit agency to allocate its service data across the UZAs and rural areas served by the transit agency (consistent with the UZAs that the reporter described on the B Forms). This form is critical as it is the basis for how FTA apportions formula grants.

11 The following is a link to the USOA: https://www.transit.dot.gov/sites/fta.dot.gov/files/docs/ntd/56681/uniform-system-accounts-usoa-effective-fy18_0.pdf.

https://www.transit.dot.gov/sites/fta.dot.gov/files/docs/ntd/56681/uniform-system-accounts-usoa-effective-fy18_0.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/docs/ntd/56681/uniform-system-accounts-usoa-effective-fy18_0.pdf

ix. Declaration Forms (D Forms) There is only one declaration form (D-10). The purpose of this form is to ensure that the reporter’s Chief Executive Officer (CEO) or CEO designate certifies the validity of the data submitted to the NTD. For instance, typically a staff member of a transit agency may enter all the data. However, the NTD requires a high-level executive sign off on the validity of the data for purposes of accuracy and completeness using this form.

x. Monthly Forms (MR Forms) There is only one monthly form (MR-20). The purpose of this form is to gather current service supplied (e.g., vehicle revenue miles) and service consumed (e.g., ridership) data. It is completed monthly and provides a current look at the state of transit in the country.

xi. Safety and Security Forms (S&S Forms) There are four S&S forms (S&S-20, S&S-30, S&S-40, and S&S-50). The purpose of these forms is to gather safety data. This data supports FTA’s safety office (TSO) and its State Safety Oversight Reporting tool (SSOR).

The S&S-20 and S&S-30 forms are completed once annually. The S&S-50 form is completed monthly. The S&S-40 form is completed separately for each safety event that meets certain thresholds. Some agencies do not fill out any S&S forms each year. The largest transit agencies can fill out as many as 1,300 per year.

4. General Data Validation The contractor performs validation across all CLINs. Validation consists of examining the completeness and accuracy of the data submitted to the NTD. The purpose of validation is to ensure that the data is complete, accurate, and is consistent with the definitions and standards established in the NTD policy manuals and Uniform System of Accounts.12 Invalid data is regularly submitted to the NTD for a number of reasons including:

• Typographical / data entry errors

• Incorrect assembly of the data from the agency’s record keeping systems by the individual responsible for submitting the data to the NTD

• The agency’s underlying record keeping systems not keeping the data in accordance with

NTD definitions and standards

Many of the NTD’s validation checks rely upon checking that the submitted value falls within in an expected range. FTA has several ways of defining the expected range which are as follows:

• Comparing the submitted data with data submitted for an earlier period. For example, it’s unusual for transit operations to change by more than 20% from one period to the next, outside of unusual circumstances, such as a global pandemic;

• Comparing the submitted data for one field with data submitted for another field in the same report (for example, if vehicle revenue miles increase by 10%, FTA would expect vehicle revenue hours to also increase by about 10%); and

12 See e.g., https://www.transit.dot.gov/ntd/2020-ntd-reporting-policy-manual.

https://www.transit.dot.gov/ntd/2020-ntd-reporting-policy-manual

• Comparing the data to industry benchmarks. For example, dividing vehicle revenue miles by vehicle revenue hours calculates average speed. It would be nearly impossible for average speed to exceed a typical speed limit.

FTA’s validation checks can be thought of as producing two types of errors as described below:

• Type 1 errors occur when the validation rule activates an alert. For example, a user submits data that activates an alert (or ‘check’). The system then requires the user to provide a written explanation that was entered correctly despite the alert. There is a subset of Type 1 errors that occur because the data in the prior period was incorrect, thus producing an unusual percent change.

• Type 2 errors occur when a validation check does not fire, even though the data was either entered incorrectly, or else does not meet NTD definitions and standards. Type 2 errors are hard to measure. However, Type 2 errors can be identified retrospectively – for example, when the subset Type 1 errors described above occur. One example of a Type 2 error would be where a transit agency reported 0 for February and March ridership even though service restarted in March. A validation check for March would not catch this as it was consistent with the prior month, but in reality, ridership should not have been 0.

FTA seeks continuous improvement in its validation procedures to minimize both Type 1 and Type 2 errors. In total across all CLINs, the contractor indirectly or directly manages approximately 1,500 business validation rules. The contractor indirectly manages 1,100 of the 1,500 business rules as these rules operate in the Appian production environment. The contractor is responsible for maintaining general documentation about the business logic and submitting requests to TAD to update, fix, or change these rules. The contractor also directly manages 400 of 1500 validation rules, including documentation and SQL code. FTA intends to move these 400 validation rules into the Appian environment over time.

The NTD program also assures valid data submissions through audit requirements. Specifically, the NTD requires each agency to periodically submit sufficient documentation that their NTD data is audited. A full description of audit-related documentation is listed in the NTD’s Annual Policy Manual.13 The contractor reviews the submitted auditor’s statement for completeness and corrective actions related to negative findings.

ANTICIPATED BASE PERIOD: November 1, 2022 – November 30, 2023

1. CLIN 0001 – NTD ASSISTANCE FOR 11/2022 – 11/2023

The tasks associated with this CLIN are: (1) operate the NTD Help Desk, (2) resolve applicable help desk tickets accurately, (3) update website resources, and (4) provide reports associated with this CLIN.

FTA requires that the contractor provide assistance to transit agencies to resolve inquires and to improve the quality of reported data. The contractor is required to log, categorize, and resolve those tickets that require NTD business-knowledge—referred to as ‘Tier 2’ tickets.

13 See https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf.

https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf

Tier 2 tickets come in in one of two ways. First, the contractor will receive some Tier 2 tickets via emails to NTDHelp@dot.gov directly from transit agency personnel. Second, some Tier 2 tickets originate as phone calls to FTA’s overarching IT Helpdesk. The FTA IT Helpdesk is led by a separate office at FTA. The FTA IT Helpdesk resolves tickets that do not require business knowledge; however, the IT Helpdesk will route those tickets that do require business knowledge to NTDHelp@dot.gov.

The contractor is required to provide assistance from 8:00AM to 7:00PM Eastern Time.14 This includes managing inbound and outbound communications from customers. Inbound communications consist of emails, while outbound may involve either email or phone call. Most inquiries requiring programmatic assistance will be received via email and require outbound email or phone communication. The contractor will not be required to host an inbound public-facing phone line to provide technical assistance (this is handled by the FTA IT Helpdesk).

The contractor’s responsibility is to provide assistance to the individual or entity that submitted the ticket—typically transit agency personnel. To do so, FTA requires that the contractor log and track tickets using whatever incident tracking tool FTA uses (currently JIRA software package by Atlassian). FTA uses this software for assistance requests, software issue tracking, and project management. JIRA licenses and access will be provided by FTA at no cost to the contractor.

The contractor is required to resolve all Tier 2 tickets – answering their question or ensuring their issue is addressed in a reasonable amount of time. FTA requires that the contractor makes initial response within two business hours. FTA expects all tickets not raising novel questions regarding interpretation of reporting requirements to be resolved by the following business day. Overall, FTA expects 33% of all Tier 2 tickets to be resolved within 1 business day and 90% within 4 business days.

FTA requires that the contractor use DOT email addresses when engaged in government business with persons outside the agency. DOT email addresses are supplied by FTA. However, FTA requires that the contractor uses their own phone numbers to make outbound phone calls to handle Tier 2 tickets as needed to close out tickets. FTA also requires the contractor to use JIRA (or whatever incident tracking tool FTA IT uses) to document, tag, prioritize, and appropriately route all tickets.15

i. Manage Tier 2 JIRA Tickets The contractor is required to manage and resolve approximately 1,000 Tier 2 NTD JIRA tickets per year.16 This includes all aspects of work, from logging the initial inquiry in JIRA through resolving the issue and sending the customer an acknowledgement that their question has been

14 The FTA will provide the contractor with access to manage the NTD’s incoming email address: NTDhelp@dot.gov at no cost. In addition, routine communications for validation (described in CLIN 0002 below) may require phone support to assist NTD reporters in meeting their deadlines.

15 For instance, FTA requires that the contractor prioritizing time-sensitive questions and questions with funding implications.

16 This RFP refers to Tier 1 and Tier 2 tickets according to NTD program terminology and described herein. Other FTA offices (e.g., TAD) use different terminology to refer to what the different tiers mean.

http://uscode.house.gov/uscode-cgi/fastweb.exe http://www.uscis.gov/ mailto:NTDhelp@dot.gov resolved. Requests range from simple to complex. Complex requests are typically referred to FTA staff for an official decision, and so may take longer to resolve. FTA requires that the contractor is responsible for deleting duplicates and prioritizing Tier 2 tickets.17

Of the 1000 Tier 2 Tickets, 200 are complex, which involve ‘policy issues’ and typically require resolution by FTA staff. Policy issues are those that affect multiple agencies at once rather than specific agency-issues, which are typically handled through the Appian process and validation (see e.g., CLIN 0002). Examples of complex policy issues are as follows:

• Clarifying to multiple agencies about how to report unusually configured large transit facilities and whether these count as a single facility or two facilities;

• How to clarify when an agency’s project has entered the “Engineering and Construction” phase, requiring certain NTD reporting;

• Answering questions about the sufficiency of limited audit documentation;

• Approving new methods for counting vehicle revenue miles;

• Intaking and routing ‘bug’ tickets from customer reports into JIRA for TAD developers to fix;

• Handling NTD exceptions (described further below).

The issues above are policy issues rather than issues that apply to one particular agency. The Appian system handles issues specific to one particular agency. However, in some cases, an agency will email validation questions to NTDHelp@dot.gov, which are covered under this CLIN. In those cases, TAD’s staff under CLIN 0001 will route the inquiry to the applicable analyst under CLIN 0002 (Annual Module).

Of the 1000 Tier 2 tickets, 800are less complex. Most of the 800 less complex tickets fall into one of the following categories:

Data Collection Guidance FTA receives approximately 100 requests per year for general data collection questions. These inquiries consist of questions regarding different types of forms and individual form fields. Some of these questions come in directly from the NTD reporters or FTA staff. For instance, an FTA regional staff member may ask that the contractor help answer hypothetical questions, such as:

17 While not final, FTA is considering adopting a JIRA categorization framework that delineates Tier 2 tickets along the following lines:

(1) The question can be definitively answered by referencing the existing NTD Policy Manual— https://www.transit.dot.gov/sites/fta.dot.gov/files/2020- 10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf

(2) The question can likely be answered by referencing the existing Policy Manual, but there is some ambiguity that requires confirmation from FTA staff, or

(3) the question raises a novel issue that requires FTA to make a policy decision. These decisions often set new precedents for use in subsequent Tier 2 tickets.

mailto:NTDHelp@dot.gov https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf https://www.transit.dot.gov/sites/fta.dot.gov/files/2020-10/2020%20NTD%20Reporting%20Policy%20Manual_1.pdf

“A current transit agency is considering acquiring a new bus that would be used for both Demand Response and Fixed-Route service. How should the agency report this to the

NTD?”

FTA requires that the contractor is responsible for maintaining staff knowledgeable to answer similar questions.

Data Product Guidance FTA receives approximately 50 requests for data product guidance per year. These inquiries consist of answering questions regarding where to find various data points across the NTD’s data products.

New User Management FTA requires that the contractor process requests from transit agencies that wish to newly begin reporting to the NTD (‘New ID’ requests). These requests typically begin with an email from the agency to NTDHelp@dot.gov with some limited information. The contractor will be required to do applicable follow-up if information is missing or needed. The contractor then creates a JIRA ticket from that information for FTA approval.

Automatic Passenger Counter (APC) Requests

FTA requires that the contractor also answers requests for Automatic Passenger Counter (APC) certification. In the past, FTA has received 1 to 20 requests for new APC certification annually.

Processing these applications involves reviewing the application for consistency with NTD reporting requirements. In particular, transit agencies planning to use APCs must document how they will validate the accuracy of the APCs, and how they will handle equipment errors and malfunctions. In most cases, approval requires requesting additional information from the transit agency on the exact nature of the service provided. The contractor is responsible for responding to APC requests both for new APCs and for modifications given COVID-restrictions.

ii. Updating NTD Frequently Asked Questions Website

The contractor will be responsible for updating FTA’s NTD Frequently Asked Questions (FAQ) website content quarterly. This includes updating the FAQ with new entries, consolidating, or removing previous entries, and describing how the updates allow users to answer Tier 2 questions on their own more easily. The current NTD FAQ website is here:

https://www.transit.dot.gov/ntd/ntd-reporting-frequently-asked-questions.

iii. Monthly Status Report FTA requires that the contractor produces a monthly status report to FTA. Specifically, FTA requires that the contractor keeps written documentation summarizing data in a manner that answers changing business questions. Example questions include: ‘how many total inquiries were received by the NTD Helpdesk email address,’ ‘how many Tier 2 tickets [customer inquiries] were answered in a given time period,’ ‘how long each inquiry took to close,’ and ‘how many inquiries were received by category.’ The report is 2 to 3 pages and contains a table with approximately 20 different inquiry categories as rows, counts by category, average hours to resolve the request, and percentages that were resolved within different SLA timeframes. In addition, on an as-needed basis, the contractor will report out the results within FTA and any mailto:NTDHelp@dot.gov https://www.transit.dot.gov/ntd/ntd-reporting-frequently-asked-questions observed trends. This support could also include writing observations or recommendations for improvements to reduce the number of incoming tickets.

2. CLIN 0002: ANNUAL MODULE AND VALIDATION FOR 11/2022 – 11/2023

The tasks associated with this CLIN are: (1) data collection, (2) validation, (3) other support; (4) failure to report escalation; and (5) annual validation improvement summary report.

During the base period (November 1, 2022 – November 30, 2023), the contractor must ensure that annual data is collected and validated so that the data is complete, complies with published reporting requirements, and finishes on-schedule. Most of this work is performed by contractor’s staff in Appian.

The work involves resolving approximately 60,000 ‘issues’ in Appian, which field entries from reporters that were anomalous for inconsistency with the 1,100 in-system and 400 branch validation rules. Each day the contractor disseminates a validation log among its staff to track and resolve a subset/caseload of the 60,000 issues that were submitted by reporters from the previous day. The yearly work is tackling the 60,000 and closing them all out by of the NTD report year.

i. Data Collection.

The contractor is responsible for managing all aspects of collecting NTD’s annual data.18 The reports are typically due four months from the end of the agency’s fiscal year. NTD reporters use FTA’s Appian website to enter their annual data. The data are entered as annual reports and FTA is responsible for signing and closing the review of each report. The contractor is responsible for distribution of report caseload to contracting staff.

Nevertheless, the contractor is required to track and resolve the 60,000 report-data quality issues, with appropriate communication inside and outside of Appian.

Work inside Appian refers to Appian system tasks.19 Appian system task management is iteratively performed throughout the data collection process; reports may be revised but are ultimately routed to FTA for final approval (closeout).

Work outside of Appian means 1) performing and logging the actual email or telephonic communications, and 2) processing issues identified via the Validation Log (see Branch Supplement to Appian Validation Rules below). FTA requires that the contractor’s staff process lists of validation checks in the validation log (described below). The validation checks are commonly flags asking transit agencies to explain each anomaly. The contractor tracks the back-and-forth communications to identify whether the data point reported was erroneous or is in fact correct. The resolution of these communications is then added to the Validation Log.

18 Note, the exception is that FTA’s IT office is responsible for operation, maintenance, and enhancements of the NTD Production database (Appian), and FTA requires that the contractor is not responsible for this work.

19 Examples can be found here:

https://cms7.fta.dot.gov/sites/fta.dot.gov/files/docs/RY%2016%20NTD%20Validation%20Module%20Guide.pdf https://cms7.fta.dot.gov/sites/fta.dot.gov/files/docs/RY%2016%20NTD%20Validation%20Module%20Guide.pdf

Validation Log

An NTD ‘Validation Log’ is Microsoft Excel spreadsheet and a core component of the Annual Module (CLIN 0002). While the logs are consolidated documents, one per NTDID, they are generated via two processes: (1) Appian; and (2) Branch.

Appian and Appian Validation Rules First, Appian is the system of record for the logs. Appian has Microsoft Excel export functionality, and FTA requires that the contractor use this functionality to build, export, and re-upload completed copies of the Validation Log for processing and recordkeeping. The Appian system automatically populates most records requiring agency follow-up in the Validation Log.

Specifically, Appian records which of the 1,100 of its in-system validation business rules an agency triggered when they submitted their annual report. In the vast majority of annual reports, data submissions only trigger a small fraction of rules. A typical annual report triggers 50 validation checks, but that can range from 0 to 200.

Branch Supplement to Appian Validation Rules The contractor supplements the Validation Log with the results from the additional 400 Branch environment validation checks. These are rules that are not all in Appian due to one of several reasons, e.g., performance impacts. The automated portion of this process is first. FTA requires that the contractor uses nightly database stored procedures to produce results in a similar file format to the Validation Log.

FTA requires that the contractor’s staff use these files to transcribe the results into Appian so that Appian contains a full record of validation rule results. For instance, if a given agency had 50 of the 1100 validation ‘in-system’ rules trigger, 18 of the 400 Branch validation rules might have also triggered, for a total of 68 records in the final Validation Log. After the extra 18 validation rules have been entered into Appian, FTA requires that the contractor then exports the Validation Log for its staff to complete.

Final Completed Validation Log To complete the Validation Log, FTA requires that the contractor communicates with each agency to make corrections or document explanations. Corrections account for errors while explanations occur when the agency’s submitted data was anomalous but correct.

Annual Module Schedule

FTA has established the targets shown in Table 2 for completing the validation process each year. The annual reporting cycle is described further in the NTD Policy Manual.20 The contractor must develop and execute a written performance work plan, so that these reporters are fully validated (‘closed-out’) according to the table below. The closeout-out process is also described further on page 21 below. The contractor must complete validation of all annual reports by July 7th of each year including processing all data submitted into Appian. FTA will receive the data no later than July 7th and complete its approval by July 15th even if the interim

20 The reporting manual and uniform reporting requirements can be found here:

https://www.transit.dot.gov/sites/fta.dot.gov/files/2021-09/2021-NTD-Full-Reporting-Policy-Manual_1.pdf targets were not met.21 Currently, FTA relies on the contractor to recommend and implement escalating consequences for non-responsive agencies (described further below in CLIN 0015).

As part of meeting the schedule below, FTA requires that the deactivate non-responsive agencies from the NTD.

Table 2: When Annual Reports are Due

Fiscal Year End Date

Report Due Contractor Work Complete -

Final Closeout Date

FTA Final Closeout Date

# of Reporters

June 30 or earlier

October 31st March 15th March 22nd 60% (1,800 of 3,000)

July 1 to September 30

January 30th May 15th May 22nd 20% (600 of 3,000)

Oc…