2-9--ADOT_ECD_DLSSU_RFP--NEC_Migration_Process_FINAL_2022-04-18.pdf
PDF 212 KB Posted
- Attached to
- Master Agreement AZ|CTR061345
- Contract number
- CTR061345
- Issued by
- Arizona
About this file
This document is an NEC Response to the Arizona Department of Transportation (ADOT) Enforcement and Compliance Division for a Drivers License Security Software Request for Proposal (RFP), specifically addressing the Migration Process section. The document involves NEC as the existing vendor and ADOT as the government agency, focusing on a detailed migration strategy for transitioning facial recognition system (FRS) data and services.
The response outlines a comprehensive five-step migration process for transferring driver's license gallery data and case investigation information, including data extraction, validation, template creation, delta loads, and final reconciliation. NEC proposes exporting biometric data using NIST Electronic Biometric Transmission Specification (EBTS) files and will provide continued maintenance and support through the transition period. The document also specifies NEC's expectations from ADOT, including assistance with problematic records, bearing costs for continued support, and managing vendor interactions during the transition. NEC will provide transition planning assistance and is prepared to accommodate potential non-standard export requests, with the understanding that additional development efforts would be at ADOT's expense.
View the file
Other files
Show all 28
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
An NEC Response to the Arizona Department of Transportation – Enforcement and Compliance Division ECD Drivers License Security Software RFP – BPM004292
Section 9 – Migration Process
Page 9-1
9 Migration Process RFP Reference: APP Portal, Financial Questionnaire, Proposal Requirements for Drivers License Security Software Overall Solution, Migration Process
9.1 Initial Migration Services
NEC has extensive experience in data migration and synchronization of existing biometric data between versions of our FRS solutions. We will follow the same proven migration process to seamlessly transition services between systems. The following is a walkthrough of each main step in the proposed data migration method.
9.1.1 Step 1: Data Extraction from the Existing NEC FRS
* * * B E G I N C O N F I D E N T I A L I N F O R M A T I O N * * *
As the existing FRS vendor, NEC will directly export all facial data including the DL photo gallery with associated demographic data as well as the historic investigation case database. If ADOT chooses the mugshot option, a parallel migration effort will also be used to import the historical mugshot database through an export/import process. With our extensive experience in data migration, NEC has a suite of data export and loading tools that support a variety of data formats, including those described below. NEC will work with ADOT to determine the best approach for exporting data as part of the requirements review process for each migration effort.
• Database Views – Generally the simplest and most direct way to migrate data is for the existing NEC FRS to provide a direct database view to the biometric images for migration. For this approach to work, both demographic metadata and biometric image data must be stored within the database structure. Information required by the view or views are Person ID, Biometrics ID, face images, and a last-updated timestamp. The system transfers data directly into the cloud FRS database using a set of staging tables to store and validate the data.
• NIST Files – With the NIST file approach, individual records are extracted from the existing NEC FRS to a file system location. The NIST packaging allows for single file containerization of both demographic and biometric data on a per-person basis and means that a direct connection is not required between the system and the new cloud system. Upon receipt of the exported NIST files, data is initially loaded into a set of staging tables within the NeoFace FRS database to allow for data checks and validation.
• Image files with data in CSV format – If it is not possible to extract the data in NIST format, NEC can also support image files with a CSV file that provides demographic and reference information. In a similar fashion to the NIST files, this information is loaded into a set of staging tables within the system.
Based on one of the extraction formats identified above, NEC loads the NeoFace FRS with the biometric and demographic data into a set of staging tables as the first landing zone for the data.
Page 9-2
9.1.2 Step 2: Data Validation and Transformation
Once data has been loaded into the staging tables, an initial data check and validation can be performed. The benefit of using the staging table approach is that potential errors in indexing or formatting can be fixed without rerunning the extraction or initial load process. Tables are also formatted without the full database constraints, so anomalies can be identified. Additionally, a data transformation for date formats or lookup codes can also be applied to smooth the final load process.
A technical report is created for ADOT with any anomalies identified in the validation of the exported data.
9.1.3 Step 3: Template Creation and Loading
Template creation includes the processing of images for facial detection and generation of matching templates (the internal representation of images). As NEC has already extracted facial templates from the existing database there should be none, but there could be a very small percentage of processed images for which a template cannot be created due to differences in the algorithms. These will be placed in an error queue and reported to ADOT for further review.
During data migration, the thresholds used to verify the quality of the images are set to relaxed values during the migration and cleanup process to get the majority of the images loaded into the system and statistical information collected based on the ADOT data set. The results of the statistical analysis form the settings used for face quality once the system goes live.
9.1.4 Step 4: Delta Loads
This will take place after the initial migration and until the final switchover to the production system. A series of delta loads is performed to bring the FRS data back in synch with legacy NEC FRS and process any newly added biometric data. Each delta load follows the same processing steps of the initial migration using the already-used methodology.
9.1.5 Step 5: Final Reconciliation
At the end of the conversion process, registration to the legacy NEC FRS is turned off and one final delta load process is run to transfer all remaining data. The process includes a final conciliation between the new system and the ADOT database to ensure consistency between the legacy NEC FRS and the NeoFace FRS solution.
* * * E N D C O N F I D E N T I A L I N F O R M A T I O N * * *
9.2 End of Contract Migration Services
NEC would work with ADOT to determine which services are needed from NEC by ADOT for smooth transition to another platform, and NEC would then provide a fair and reasonable quote for those services. NEC-provided services for transition can include but are not limited to those described below.
Page 9-3
* * * B E G I N C O N F I D E N T I A L I N F O R M A T I O N * * *
9.2.1 Continued Maintenance and Support Through Transition
NEC will continue to provide support and maintenance of day-to-day system operations following the termination of the existing Support and Maintenance Contract up to the point when the system is scheduled to be uninstalled.
9.2.2 Data Migration Export
NEC understands how much time and energy goes into the creation and maintenance of identification system databases and how important it is that effort and information are not lost when transitioning to a new platform.
There are two general types of data stored in the NEC NeoFace FRS: DL gallery data and case investigation data. DL gallery data is the biometric information and demographics captured as part of the DL or ID application process and is used for identification. Case investigation data includes the investigative probe image data, matching results, and the associated witness sessions. How the two types of data are exported is handled a little differently, but the overall flow is similar.
NEC exports biometric data as NIST Electronic Biometric Transmission Specification (EBTS) files, which are the widely used standard that most biometric vendors (including NEC) should easily recognize and be prepared to accept.
Case investigation data covers a wider range of information and not all of it may need to be transitioned to the new system. Final export formats will be agreed on between NEC and ADOT, but by default files are exported as files in the same format they were uploaded to the system. Metadata for these files and associated results are exported as a database importable text file (CSV) that includes the name and location of the exported file, in addition to the other metadata.
The overall flow for how these data extracts occur is the same:
1. An initial export is performed of all data.
2. Since the system will likely continue to operate while the new platform is implemented, there will be one or more “catch ups” that will export only new or changed information since the prior export and that will identify records previously exported that were deleted since the prior export. Catch-ups will be done until one final catch-up is performed immediately before switchover to the new platform.
After each export, reports will be produced that indicate the number of successfully exported records. For catch-up conversions, the reports will also tally changed and deleted records.
NEC will also create a report on error records that could not be exported, which will include the reason for the error.
Some of these errors will be correctable, whereas others may not be. Correctable errors on the list may then be fixed so they can be exported on the next catch-up. Correctable errors may also require ADOT assistance and research to fix.
While the above method is the default means of exporting data, NEC may be able to accommodate some non-standard export requests such as performing a data translation of some types of information to a non-standard format. These types of requests would require additional time and effort to develop and implement, and the costs associated with the additional time and effort would need to be borne by ADOT. Also, some types of requests might
Page 9-4 not be feasible because of time constraints, prohibitive costs, or asking for a translation to a non-NEC-proprietary format.
9.2.3 Transition Planning Assistance
NEC will provide the necessary information for the timeframes, dependencies, etc., for services it performs for use by ADOT in creating ADOT’s own overall transition plan.
9.2.4 NEC Expectations from ADOT
1. NEC expects ADOT to assist with problem records encountered during data migration export that may need record research by ADOT to resolve.
2. NEC expects ADOT to bear any costs for continued NEC support or services, past the contract termination date, at the then current monthly maintenance rate, until the final transition has occurred, and the NEC system is turned off.
3. NEC expects ADOT to be the overall manager and contact point between the vendors involved in the transition.
* * * E N D C O N F I D E N T I A L I N F O R M A T I O N * * *
| 9 Migration Process |
| 9.1 Initial Migration Services |
| 9.1.1 Step 1: Data Extraction from the Existing NEC FRS |
| 9.1.2 Step 2: Data Validation and Transformation |
| 9.1.3 Step 3: Template Creation and Loading |
| 9.1.4 Step 4: Delta Loads |
| 9.1.5 Step 5: Final Reconciliation |
| 9.2 End of Contract Migration Services |
| 9.2.1 Continued Maintenance and Support Through Transition |
| 9.2.2 Data Migration Export |
| 9.2.3 Transition Planning Assistance |
| 9.2.4 NEC Expectations from ADOT |
File details come from the government source that posted it. Updated .