NTD_Preproposal_Conference_Transcript_-_DTFT6017R00001_Amendment_0001.pdf

Pre-Proposal Conference Presentation Transcription Event Date: Monday, January 30, 2017

Scheduled Time: 10:00 AM to 12:00 Noon EST

>> Stephanie Cooke-Nwadibia: Good morning, everyone. Welcome to the National Transit Database Validation Services Pre-Proposal conference. I am the contracting officer for this procurement. Welcome to those also on the webinar. First, we will start off with a disclaimer for the presentation. Please keep in mind this information is for informational purposes only as well as clarification. Also, in the event that there is a discrepancy with the slides, the RFP will take precedence. Everyone by now should have received a copy of the agenda as well as the PowerPoint presentation slides.

For those on the phone via webinar, you can see it as well. Once again, I am Stephanie Cooke-Nwadibia, the Senior Contracting Officer for this procurement. This is Maggie Schilling.

>>Maggie Schilling: I'm the Program Manager for the National Transit Database

>>Stephanie Cooke-Nwadibia: This is the Contract Specialist, Shequesha Evans.

Also on our team we have Steve Pereira the Attorney Advisor, Office of Chief Counsel.

With our agenda this morning, of course, we have two hours allotted for the time period and hopefully it won't take two hours. We are going to go over a summary of procurement, proposal submission requirements, evaluation criteria, special requirements that are in section H of the actual RFP presentation, an overview of the program so you can get a better idea in terms of what your firm may be submitting a proposal for, introduction of course, to the NTD reporting system and a question and answer session. Please keep in mind the questions were due on Wednesday January 25 so the questions that we are going to be answering today are the ones that have been submitted that we've already prepared responses for.

Now, I will present a summary of this procurement. The solicitation was issued on January 12. It is a total small business set-aside based upon our market research we decided to do a total small business set-aside. The incumbent contractor is Boyd Caton Group and the total estimated value is 19.9 million. Also based upon market research we did a request for information on FedBizOpps. I'm not sure if most of you have seen it a couple of months ago. We came up with the NAICS code that we thought would apply better to this procurement 541611 as a result. Our anticipated contract date is late summer or fall of this year. Proposals are due March 6, 2017 at 3:00 p.m. Eastern time. Our period of performance is five years. Therefore, one year base period and 4 one year option periods. The contract type, if you looked in section B and C you saw that we are doing Firm-Fixed-price and Labor Hour CLINs. The travel CLIN is cost reimbursable and we are doing performance-based contracting. For this procurement, we have a Statement of Objectives in Section C and your firm will be submitting a performance work statement in accordance with the evaluation criteria in section M.

Instructions to offerors concerning the submission is available in Section L which contains the proposal submission requirements.. There are five volumes which must be submitted regarding your proposal to the Government in response to this solicitation.

In the posted solicitation, we did notice that there was a discrepancy for volume three past performance where it says one electronic copy/e-mail copy so via an amendment to the solicitation we are going to take that out so there will be no e-mail copy. Just three hard copies, one original copy and one electronic copy that should be delivered with your proposal by March 6. Also, we had noticed that there is a section about certification of data in section L.9 as well as section K, Reps and Certs. Therefore, we need those parts actually submitted with the actual proposal.

So let's go into actual evaluation criteria. A brief overview . . . For the actual evaluation criteria we have technical ability and approach which is 80 points max. Within that actual technical factor we have three sub factors. The first factor, of course, being performance work statement and quality assurance surveillance plan which is worth 40 points. Then we have key personnel and staffing which is a total of 25 max points.

Then another part, oral presentation. The second part was past performance, 20 points max. Two Sub factors listed on the slide and then we also have Price which is not a weighted factor, but also part of the actual evaluation. One thing I wanted to pinpoint about price is that we are actually going to be evaluating for completeness, reasonableness, and balance. Please ensure that you are submitting your best proposal to assist us with making a best value determination.

For the actual technical evaluation, we are looking for you to actually address your objectives which are actually in the Statement of Objectives. The deliverables as well as the timeliness are critical aspects to address with your proposal. It's also important you are clear and concise and detailed in your response. We received a question in terms there was a part in there about-- someone had asked what we met by please not just repeat what is stated in the RFP. Therefore, I am stressing that when you are submitting your proposal response, please do not just repeat exactly what is in the statement of objectives as saying, for example, my firm can do this. We need you to demonstrate to us that your firm can do this.

Also, as part of the technical evaluation, you have to submit, like I had said earlier, the performance work plan and quality assurance plan. With this, it must demonstrate of course like I said previously, that you can meet the required objectives and like I mentioned a minute ago, don't just repeat the RFP. Also one thing I was going to say that is not on the slide, just overall, please don't put a lot of fluff in your proposal submission because we can read through the fluff. You have a max page limitation limit per volume so please use it wisely. Also as part of that actual sub factor, we are looking for you guys to provide innovative approaches, not just saying you have an innovative approach. Demonstrate your innovative approach. Also as part of the technical evaluation, we have received a question about what were the actual key personnel. Because in one section it may say project manager and subject matter expert and the advisor. So we actually have three labor categories that are key personnel. Also one thing I wanted to point out is we want resumes for all personnel, not just key personnel. Within that section we have labor category descriptions with qualifications, please make sure when you are submitting your resumes that they are actually the personnel that actually meet the qualifications, not something you just believe they do or you are not sure. Because of you think about it, you are putting your best proposal forward.

Okay, oral presentations. Presentation should be no more than 30 slides. You have 45 minutes allotted for it so we are actually going to schedule it for a whole hour. After the 45 minutes, you will have 15 minutes for questions and answers. For this presentation, please make sure that you have a maximum of three presenters, no more than that. Also for scheduling purposes, and room availability we cannot accommodate you if you are going to bring six or seven people with you that are not presenters.

Please be cognizant of that when determining whom will attend the presentation. As part of the oral presentation, offerors must address some evaluation criteria stated in Section M of the solicitation. . There are three areas concerning evaluation for this oral presentation sub-factor that you must address. Please make sure you address those in your slides that are submitted to us that you will be actually presenting at a later date.

Now, we are at special requirements and Maggie will be talking to you about that.

>>Maggie Schilling: Good morning. Again, I am Maggie Schilling, Program Manager for the NTD.I will walk you through the special requirements and an overview of the NTD program. I will also show you the reporting system that we currently use.

You will not have access to the system after this demonstration. It is just for reporters and this will be your one chance to see before you put a proposal forward.

Okay, special requirements. Please note that FTA will not provide a building or office space for this contract. This is an off-site contract so you will have to provide your own space and building. You will provide DOT access to the facilities where your contractors are located. And you will provide us with any documentation, databases or personnel used in the performance of the contract anytime we ask for it.

All right, special requirements for IT. FTA maintains an agreement with a platform as a service provider. Appian is our provider right now for the reporting tool and platform.

You will be given access to the system. We will provide you with VPN access and access the back end data. You will be providing the computers for your staff members and all of the equipment. FTA will retain all rights of ownership to the NTD data, data products, scripts and code anything you create under contract for this is owned by the FTA. Any time we ask for it needs to be provided to us. At the end of the contract everything created will return to us. And you are not to redistribute or use that in any way without permission from us. You cannot release data products without our permission, they go out on a schedule and they can't be given out early. So this is just to remind everyone FTA pays for it, FTA owns it.

Okay, so I will now provide an overview of the NTD program. The NTD program is actually codified in transportation law. It is section 5335 and it authorizes the Secretary of Transportation to maintain a reporting system using a uniform systems of accounts to collect public transportation data. This applies to beneficiaries of FTA formula funds. I don't know if you're familiar with Federal Transit Administration we give out roughly $6 billion in formula grants. The formulas are determined by Congress and the NTD is what feeds the formula grants. The data we collect (in the NTD) is what helps inform how that money is doled out. We have a couple of different grant programs, but the grants programs that are required to report to the NTD are recipients of 5307, which are urbanized area recipients, so all the transit providers in large cities, or recipients of 5311, which is other than urbanized areas basically rural providers. States, tribes, small transit providers receive 5311 money. Small providers reported in to the NTD through the states. The states largely manage that NTD reporting. In 2012, with MAP-21, a third section was added to 5335 and this will be rolled out in the first year of this contract. We will begin collecting additional asset inventory data from all of our providers. This also will bring in additional reporters. We estimate 700-900 additional small providers will be reporting through the state. Essentially, the states manage a large group of sub- recipients and report on their behalf. The 700-900 will come in under states and will be new individual reporters and will be subject to asset inventory reporting.

The NTD provides input to the apportionment formula. This is the most important use of NTD data and it's one of the annual data deliverables. The annual data sets are the largest annual deliverables for the NTD. These are used by the industry for resource planning. People use them for service planning, peer comparisons et cetera. The NTD is the only database of standardized transit data in the nation so lots of institutions use it and people rely on this data such as lobbying groups for the transit industry and academic research institutions. It's an important data set for the transit industry. We also use it for government reporting. For instance we use it to feed our condition and performance report to Congress. It's a mandated report that goes out biannually and the NTD data feeds this report. The actual creation of that report is a separate contract and not part of this procurement. We also use this information for Government Performance and Results Act reporting, we are all subject to it as government agencies and so the NTD data also feeds the performance metrics.

Now let’s talk about how the NTD works. We collect information on a fiscal year basis from all transit properties in the United States. We collect a variety of data. I will repeat some of this but, repetition is probably good in this case. We collect financial information, service information including ridership. We collect information on a few metrics on maintenance performance so maintenance failures information and asset inventory information. Everyone reports based on their own fiscal year. We have three reporting deadlines. If you're operate on the old federal system June until July your reports are due to us by October 31. If you have a fiscal year that follows the new federal fiscal year October to September you have to report to us by January 30th. And if you run on a calendar fiscal year you report to us by April 30th.

Essentially, this means that four months after your fiscal year ends you report your data to the NTD. The data sent to us, we validate but we do not audit. We do not go into the books and look against what everyone has in their own financial systems to make sure they reported it to us correctly. They are self-certified. They are required to provide and independent audit every single year that says, yes, this information is correct and it can be mapped to your uniform system of accounts. So to back up a little bit, the NTD is organized by uniform systems of accounts which is the accounting structure that's available on our website. I will give you that web address. The two pieces of information that will be most helpful are: the policy manual and the uniform system of accounts.

These are the two most important pieces of information to understand how the NTD runs, what the policy is, how we organize information etc. These will give you a good jump start.

Once they report to us, their CEO will certify that the report is correct. They tell us: “this is correct, you will give us money based on this and we will be held responsible if we are not honest about the information provided to you”.

>>Action in audience: Someone raised their hand in the audience and some stated “there is a question from someone in the audience”. Maggie Schilling stated “Let's save that until the end if you don't mind. All right, so what is reported? As I said agencies have to report all transit services, both federally and non-federally funded. It doesn't matter if you take money from us for the service. All service you provide has to be reported to us whether or not we directly funded it. It's easier to convey that message to agencies. All services urban, rural or both, and some agencies operate in both environments, they report all service. One thing to note, if you provide any service to an urbanized area you automatically become an urban reporter and come into that city level reporter which is the highest level reporting. If you have fewer than 30 vehicles you qualify for a reduced report. Reduced reports and state-level reports are similar. They are smaller subsets of information that we collect. In couple of slides, you will see this as well. We have monthly reporting and if you have less than 30 vehicles you are not subject to that monthly reporting either.

>>Maggie Schilling: All right, so the three data groups in the NTD: annual, monthly and safety. If you are used to the old NTD we separated urban and rural (annual reporting). I don't separate them this way-- I am relatively new to the NTD, a year and a half. We have annual reporting. That's for both urban and rural. So all urbanized areas, or section 5307 funding recipients, they report the most information, if they have more than 30 vehicles.

Less than 30 vehicles report a similar amount of information to states and sub recipients or section 5311 recipients with no 5307 funding. We also have Tribal reporters. Tribal reporters provide the same amount of information as the reduced reporter.

And monthly we have a safety and security module so people report roll-up information and detailed incident information based on certain thresholds. So there is a safety and security policy manual as well that I encourage you to read to understand the monthly safety and security information that is collected. We collect a small subset of data monthly on ridership, vehicle revenue hours, miles, certain service level pieces of information each month and those are also for any agency that's more than 30 vehicles.

So how many current reporters do we have? As of 2015 we have 831 total urbanized area systems. 525 of those are full reports. Those are the big guys, more than 30 vehicles. About 306 have less than 30 vehicles. Those are the little reduced reporters.

Same amount of reporting as those states. The 525 full reporters are the one that also do all of the monthly reporting and all of the safety and security reporting.

The rural module is 54 states and territories and134 Indian tribes. The states report right now on behalf of just shy of 1500 sub recipients across the 54 states and territories. That number may go up about 700-900 with the asset inventory reporting.

For the urbanized areas the asset inventory reporting there may be some increase and you may have some MPOs (Metropolitan Planning Organizations) that end up reporting that were not reporting before, but by and large those large agencies will just absorb the new asset inventory reporting requirements.

Okay, so this is a tiny slide, your handout might be better. These are the annual reporting forms. As I said, we collect information on who you are, on your finances, so what you earned and then what you spent. What were your funding sources and then what did you spend your money on, what were your expenses. Transit way mileage, revenue vehicle inventory, service information, number of employees. So this is the breakdown. Full reporters are reporting almost all of these things. And your small systems and rural sub recipients you see the subset for them. They don't have separate reporting. Nobody reports except those 54 states and territories. Okay, and CEO certification, et cetera.

Okay, so every year as part of -- as I said one of the largest deliverables for this contract are our annual data products. So we make all of our data available to the public once it's collected. Virtually everything collected in the NTD is put out onto our public site and people can use it and create their own NTD database and run the data anyway they choose. These are some of the data products we currently put together.

They're all available on our website. I encourage you to look at these. I would focus on the 2015. We did a redesign of our data tables for 2015. So those are actually in the format that we anticipate using in the next contract year. When you look at them, you will see information back to the '90s, but I would focus on 2015.

Transit profiles, this is information on a provider by provider basis so everybody has a profile and it essentially a snapshot of their transit provider information, how much they spent et cetera. We do a summaries and trends document where we look at national level transit information and trends. The transit data tables, so this is from the annual module and the data tables are intended to be a digestible version of the database information. So we put raw database files out there that are just essentially Excel drops of all of the data and I don't know probably about 25 of those right now. The data tables are intended to distill that information down in a way that's usable for people, so present the information in a way that they can use. These (data products) are areas where you should include ideas for innovative approaches in your proposal.

The monthly data tables, so these come out every single month. We do have summary level information for safety and security information and for that service data we collect monthly.

And we have historical time series files. I think there is about 10 tables right now. The time series go back from 1991 all the way to the present. They are really useful products for high level information and they are good trend level documents.

Okay, so a quick summary. We have approximately 1100 reports each year. About half of those are full reporters. As I said 54 states reporting on behalf of almost 1500 sub recipients. The 700-900 will be coming in in the first year. They will be mandatory reporters in 2018. But people will be able to begin entering that information optionally in 2017 so the first year (of this contract) will be an optional year for the reporters on the next year will be a full year reporting for those reporters.

We have about 1,000 validation checks coded into the system right now. We had a couple of questions about validation checks and I'll jump into those when I get to the questions, but right now we have a set of validation checks built in and these are largely historical checks so year-over-year are you falling within thresholds or do you look totally crazy. So 1,000 of these are built into the system. They check information make sure you are entering consistently across the report and that you are not too far out of the bounds of what you entered in the past or with the industry.

We have 300 additional checks that are identified and we hope to have in the system by this fall. They either have been built and we are doing acceptance testing on them or we need to build them. Additional checks can be done outside the system. We provide a backend copy of the database nightly. You can use that to build SQL scripts that allow you to do all sorts of different validation checks against the data. We have approximately 600 reporters providing monthly reports on ridership and safety. And we have about 8,000 major event reports every single year. These are the monthly incident level reports based on certain reporting thresholds. We have just made some threshold changes to align with new safety legislation and that may increase reports about 10 percent in the coming year.

All right, so just a quick look at what the contract deliverables are for this RFP. We have technical assistance. This is having someone on duty to answer questions at all times.

We don't have a huge throughput for the help desk. I'll get into that at the end, but we do need someone to be able to answer any questions by reporters.

And then we have the annual module validation which is the largest bucket for the contract and this is looking at all the data that comes in annually and verifying that it looks reasonable.

The monthly module and the safety module validation data products come out every single month and are validated monthly and it's provided monthly.

With the apportionment files and data products those are once a year deliverables at the end of the report year. We typically get them the end of the summer. We have to have the apportionment run by the beginning of the fiscal year so there is a very hard deadline on when the apportionment files can be delivered to us in perfect shape and that is the beginning of September.

We also have an annual review meeting. It's a meeting at the end of the year where we go over together any issues we've identified in terms of validation changes to be made and policy changes that might need to be addressed et cetera. We look at high level performance against the agreed to timeline targets and performance metrics.

Then, we have reporting manuals, training materials and our website. We are expecting this contractor to do the annual updates to these products. Our website is hosted on a Drupal platform that allows us to go in and easily change content or upload the data products as they are approved. We are asking our contractor to do that under this firm fixed price CLIN.

Then we have program and performance metrics reporting. I will be asking for regular reporting against agreed to performance metrics such as how quickly are we addressing helpdesk issues et cetera. These will be regular reports provided to me on the performance against the metrics.

Travel is cost reimbursable. I think we put into the RFP what we are estimating and I believe we also put in how we estimated that. We have about 10 in person trainings every year and we like to send 1-2 analyst to those to address questions by reporters.

We have a contract with the National Transit Institute that provides that training every year so the analyst are just there as support and sometimes they will come in and help with activities.

Then we have special projects. We did have one question about special projects. We are asking for a not to exceed amount for special projects. Special projects includes things outside the bounds of our normal data products. We get questions that come in from reporters or from the front office when new legislation is being drafted. We may ask for the contractor to pull the information and present it in a way that answers whatever question we have to address.

On training provision, this is a new CLIN for this year. I get requests for a handful of trainings every single year outside of the normal 10 we have with NTI. I would like to have the ability to have this contractor provide some of this training. This requires representing the NTD at conferences or provide training on how to report to the NTD.

User acceptance testing is another CLIN where we are requesting a not to exceed amount. We are still in the process of refining our reporting system. We began collecting information in this system 3 years ago and have some ongoing user acceptance testing activities ongoing. This contractor will be doing this testing. We have an IT contractor that develops and maintains the reporting site so just to be clear, this is not building the NTD reporting tool, or maintaining the tool, it's not writing requirements or acting as the primary IT contractor. You will assist with requirements for the reporting tool, but the actual writing and documents is done by the IT contractor. The user acceptance testing that's done is really a business owner user acceptance testing so understanding how we want the system to perform and making sure what the IT contractor provides to us with this performance.

There is a plan for transition. It's affirmed fixed price and there will be three months of transition provided by the current contractor. And so we are asking for a cost of three months of transition expenses at the front end and the back end of the contract.

All right, so where can you find the NTD data products? This is a snapshot of what it looks like. The website it www.transit.dot.gov/ntd and this is where you can find all the information. On the left-hand bar is where you find the data reference materials and the National transit summary and reports we put together.

I am going to provide an introduction to the reporting system I'm going to login quickly and show you how it is organized. Our reporting tool, as I said,is on the Appian platform. As you can see, this is what our reporting tool looks like. It's a task driven system so you have a taskbar here. A task landing page here that allows you to sort and access all the tasks that are in your queue. I'm the FTA Program Manager so I have access to a couple of different tasks here. It will be slightly different for reporters and analysts that are accessing the system, but essentially if I click into a task, I have to accept it and then I can go through the task. It says here that this reporter is asking for an extension request and here's why they need an extension et cetera. The analyst has gone through it and here's the response they've provided. They will say either: we think yes, we should approve this or no, we shouldn't. These are all things the contractor should be doing through the system.

So records, let's see a report package. So that is what a reporter and an analyst will see. This is what a report package will look like to someone coming into the system.

Right now everything is entered through the front end so we have the landing page this is for the city of Grand Prairie Transportation Services fiscal year 2017 reporting and the five digit number is the reporting ID. You can click on the reporter profile. And you see here is the basic information and the users they have in the system. They do not have any reportable segments so they are a demand response only provider. I'm going to go back so I choose one that's actually -- I'll choose my old -- just because I know them the bad. I used to work with Chicago Transit Authority so I know them pretty well.

They are my go to. So the CTA you can see they have monthly ridership forms that are available. It means the report package is somewhere where I cannot access it. So here I can see the different forms they have available right now. I click into a form. Once I click it, I can view it. This is the same. If I've gone into an annual report package you would've seen a long list of annual reporting forms. So here I have all the months for 2016. We can see that those are already accepted. But what you would do is you would be able to click in and they enter their information onto the forms.

All right, so this agency is still working on their report so I can see when someone pushes their forms to the next level they are no longer viewable to me. This is versioning control through the system. What happens for an analyst, is they can see their forms by can't make changes until it's been passed on in the queue to them so as http://www.transit.dot.gov/ntd long as the task is with the reporter they are the only one who can edit the form in anyway and they submitted to the analyst in the analyst can go in and make comments, create validation issues et cetera on data items in the system.

So if I click on annual forms, you can see here is a whole list of the forms they have available for reporting. If I choose their identification form, I can view it and then I can see this is the information that they have right now, the organization type et cetera and I can update clicking in these. Each form is organized differently depending on which data we are trying to collect.

That's just a quick overview of the system. If I go through the profile, I can look at past years reports or information. I go to the form library and here I can get copies of their old reports. When people are submitting sometimes they will go through several revisions before the report is finalized and closed out so if I want to pull up different revisions I can do that in the form library or pull up their final closeout information. So when I choose the close out for 2015 these are the final forms accepted and closed out in 2015, the federal funding allocation. Right now these are dumped out to HTML for printing purposes. We are in the process of building an Excel upload/ download capability. Appian did not have this when we built the system so now it's a new plug-in.

We are looking at rolling this out for the whole system.

All right, so you can see the old Federal Funding Allocation forms and these are some of the things that help feed the apportionment formula. These are the pieces of information that inform the Federal Funding Allocation.

So let’s move to question and answer. I'm just going to go through a few of the questions that we received and answers that we have created in advance. We will also get a written version of the question and answers…

>>Stephanie Cooke-Nwadibia: We will provide an amendment to the solicitation regarding this.

>>Maggie Schilling: Okay, cool. So help desk: We had a question about helpdesk, specifically how many calls we receive a week and how long they take to address. We receive about 25 calls a week on average over the past four years and about 10-15 e-mails directly to the helpdesk. We do have a group of analysts that are each assigned to a group of reporters. Often times people will go directly to their analyst. So helpdesk is for either someone who doesn't know who their analyst is or doesn't know where to start with the NTD. So this is why you see kind of a smaller throughput to the helpdesk, because a lot of this is offset directly to the analyst groups. About 99 percent of the calls are resolved in less than 10 minutes. We have very few that take longer than that.

System bug reporting: someone asked about where these system bugs would be reported. The new contractor will be required to use the FTA change management system and the current system we use is for reporting and prioritization is JIRA. This contract does expect that the person would log all the tickets in and also assist with creating the prioritization for those tickets. FTA signs off on the final prioritization.

NTD eligibility criteria: Someone asked where they can learn more about the eligibility criteria or who is eligible to report to the NTD. Internal program documentation including SOP's will be provided to the winning bidder. We do not make those public for obvious reasons, but the policy manuals and the uniform system of accounts available on the website will help you understand who is eligible for reporting and who has to report and what they have to report.

We had questions about validation checks. Documentation of the current validation checks will be made available to the winning bidder as well. The current in-system checks rely heavily on thresholds derived from past years and industry standards.

Apportionment documents: These are considered internal documents. There are couple of places to get information on the apportionment and one place that might be most useful is looking at apportionment flowcharts on this website:

https://www.transit.dot.gov/funding/apportionments/fast-act-formula-flowcharts This is where we publish the actual flow chart of the different apportionment formulas. It will help you understand the inputs and how that information informs how the money gets doled out.

Our apportionment documentation consists of nine data sets. And then we have an adjustment memo that's 20-25 pages detailing the adjustments made to the apportionment. These are long-standing policy decisions, so a couple of very strange grandfathering clauses et cetera. This documents all the adjustments we make every year to accommodations those policy decisions.

We have a rural reporting memo right now and it's a state-by-state and is about 10 pages just looking at each states vehicle revenue miles and changes year-over-year.

We have an apportionment memo that's 275-300 pages and provides agency by agency detail looking at each individual urban agency and the issues with their validation that year, major changes and questionable data et cetera.

Separate database: We had someone ask about a separate database. So I just want to be clear again, this is not to build a reporting tool or support the build of a reporting tool.

And I just want people to be very clear we are not asking you to rebuild the NTD reporting tool in this solicitation. We are also not asking for you to maintain a separate database or have a DBA. What we will give you is a set of data. So because of federal guidelines et cetera, certain things are considered databases that you might not think of as a database. It's a dump of information into an Excel file and this could be considered a database by government standards. So what we provide to you is a backend download of the data each day. That data can be manipulated in a number of ways. We are asking is that you maintain this in a clean way and in a way that we would be able to access it at any time. We should also be able to access the scripts you are running against the data and reuse them as needed for whatever other projects we doing. The conditions of performance reports being the largest one. What we are asking is that you https://www.transit.dot.gov/funding/apportionments/fast-act-formula-flowcharts propose personnel that are sufficient to meet the requirements of the RFP and our data validation efforts. Someone asked specifically asked about a DBA. You will not be acting as a DBA for the reporting system.

Those are the questions that I pulled out to address today. Here you have the primary procurement office points of contact. You may contact Shequesha Evans or Stephanie Cooke-Nwadibia in procurement. If you have questions going forward again these -- I do not know how to say your last name. I just consistently spell it and I'm proud enough of that. It is Stephanie Cooke-Nwadibia.

>>Maggie Schilling: Okay. They are the people to contact. So I guess we are going to open it up for questions

>>Stephanie Cooke-Nwadibia: We are open to questions; however, mainly if they are burning questions you had right now to ask. Also, you may submit questions regarding this presentation as an email to Ms. Evans and I no later than 3:00 PM on February 1, 2017. Okay, I believe you had a question.

>>Vendor in audience: My name is Mark Turner, Optimal Solutions Group. My question is doing the validation checks, of course, you are looking for innovative approaches and you find gaps in the validation. Is there provision for changing the forms are changing the data I don't know if OMB clearance is required for that?

>>Maggie Schilling: So in general we cannot change what we collect without notice and comment. So we cannot add data elements. I can pretty easily remove data elements but if I am adding burden I have to notice and comment and do a paperwork reduction act. It's a big bureaucratic lift. And so it's not something that we do every single year. So it's better to sort of bundle the changes together, but in general we don't expand data collection without congressional mandate or input. And so if we are just changing the way that we collect it, like if we are changing the design of the form, that's not a problem but changes to the actual data elements is a much bigger process. Okay.

>>Vendor asking question from audience: Just a follow-up to that, so the self-validation, essentially the reporters and their self-validation, there is no third-party audit that's occurring and they are submitting that or are they purely self-certified?

>>Maggie Schilling: They are self-certified; FTA at large does triennial audits. So the triennial audit is done on all of our grant recipients and it's essentially an audit of pretty much anything that relates to federal requirement funding, reporting et cetera. But it is a broad sweep, not a deep sweep. So some NTD reporting elements have been included in the audit in the past and I think we pulled many of those out, but they can come in and out of the triennial audit process. We had a question.

Vendor in audience asked “Who is the current BPA provider?

So the current IT contractor we have a BPA so we have three providers that provide work against that BPA. Appian, the parent company, is on that BPA, Horizon Industries is on that BPA and Collabralink is on the BPA. So Collabralink is providing the O&M services for the platform. Some additional work on an inventory is done by Horizon right now, but they are all authorized to do work against the BPA. Anyone else?

>>Vendor in audience: Good Morning. I'm Mike Sperry with T Duct and I just wanted to know, are the current quality metrics being met and are there particular areas of opportunity to improve the process that you could comment on?

>>Maggie Schilling: So I don't think we can comment on that.

>>Stephanie Cooke-Nwadibia: No, this cannot be commented on.

>>Maggie Schilling: Any other questions?

>> Vendor from audience stated: You mentioned Appian, Horizon and Collabralink, but the incumbent is Boyd Caton.

>>Maggie Schilling: They are the incumbent for the contract, the other three are the IT contractors that support the reporting tool, yes.

>>Maggie Schilling: One more over here.

>>Vendor in audience: Thank you. What size of the database or all of the databases together? How many cells of data do you typically get?

>>Maggie Schilling: Oh, man, I don't know off the top of my head. I will -- I can try and include this in the final response. But our current database, so right now from the front end we are only housing 3-5 years of active data in the new reporting tool. The old one is in there in tables and then we have some very old data like from the '80s and 90s that is again just in static tables that are not query from the front end of the tool.

This is -- a different project for a different day but we would like to marry those together at some point which it will be a huge undertaking.

>>Vendor in audience: Where do you estimate the biggest level of effort to be in all of the work activities that you have laid out?

>>Maggie Schilling: Definitely the annual reporting. So that is where the most effort is expended each year, the validation of the annual reporting.

>>Vendor in audience: Thank you.

>>Maggie Schilling: Anybody else? Great. Okay. Thank you so much.

>>Stephanie Cooke-Nwadibia: So this concludes our NTD Pre-proposal Conference here today. Like I had stated earlier, if you have any additional questions please submit them to Ms. Evans and I no later than February 1 at 3:00 p.m. via an email to

us. Thank you.

>>Voice from audience: Not everyone has checked in.

>>Stephanie Cooke-Nwadibia: We checked everyone in the building lobby prior to going through security.

>>Stephanie Cooke-Nwadibia: Thank you.

>>Individual whom made prior statement about checking in: Good.