FINAL_HSHQDN-17-R-0002_RPM_RP_RFP-Atch_07-Replay_Tool_Requirements_Checklist-2017-1-19.docx
DOCX document 34 KB Posted
- Attached to
- FINAL RFP HSHQDN-17-R-0002 DNDO RPM RP
- Solicitation number
- DRAFT_RFP_HSHQDN-17-R-000XX_RPM-RP
About this file
Attachment 7 Replay Tool
View the file
Other files
Show 40 more
Latest 50 of 54 listed here. See the rest on GovTribe.
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
Radiation Portal Monitor Replacement Program
RFP HSHQDN-17-R-0002
Attachment 7: Replay Tool Requirement Checklist
The Replay Tool Requirements presented below are taken from the Domestic Nuclear Detection Office (DNDO) Replay Tool Requirements Specification.
Please respond to each requirement, using the column labeled “Y/N”, with one of the following responses: “Y” if the proposed Replay Tool meets the threshold level capability, or “N” if the proposed Replay Tool does not meet the threshold level capability. If desired, provide any comments and recommendations in the column labeled “COMMENTS.”
| ID# |
| REQUIREMENT |
| VALIDATION |
| Y/N |
| COMMENTS |
GR-1-REP
| The replay tool shall exactly reproduce the live results of the radiation detection system under test when provided the identical input data |
| Identical output, with the exception of time stamp and replay device operating system information; identification confidence within precision given in the live output file. |
| GR-2-REP |
| The replay tool shall accept as its only required inputs the input data provided by the radiation detection system along with any analysis parameters used by the detection system. |
In the event that input data must be pre-processed before analysis by the recognition algorithm, the replay tool shall perform the pre-processing.
Test that the results from the replay tool match the results from the in-situ algorithm for all verification files, as for GR-1-REP.
| GR-3-REP |
| The replay tool shall have a user-accessible facility for adjustment of algorithm parameters. |
| Test that parameters can be adjusted as documented |
| GR-4-REP |
| The replay tool shall include the same range and functionality for adjusting algorithm parameters as the in-situ algorithm on the radiation detection system. |
| Check that parameters that can be adjusted via settings on the radiation detection system can be set in the replay tool. |
| GR-5-REP |
| Reproducibility of results shall be achievable as in GR-1-REP independent of the order of processing. |
| Verify that the output (results) files are identical to the output validation files provided in the validation package when the input validation files are replayed in any order. |
| GR-6-AUT |
| The replay tool shall be a stand-alone software tool that can be installed and operated autonomously. |
Install the replay tool on each standalone (internet-disconnected) supported platform and check that the replay tool can replay all input validation files.
| GR-7-AUT |
| The replay tool shall process each input file separately. |
| Same test as GR-5-REP. |
| GR-8-INJ |
| The replay tool package shall include a translator facility to generate N42.42 files from vendor format raw data files, and translate the N42.42 files back into the raw data files suitable for use with the replay tool. The gamma-ray spectral data, neutron count data, and energy calibration data shall be stored in standard N42.42 data elements; vendor-specific auxiliary data may be in permitted extensions to N42.42. |
| Check that the translator can produce readable N42.42 files from an original raw input data file. |
Test that the replay outputs of the original raw input file and the translated raw input data file are identical.
| GR-9-INJ |
| The replay tool shall be capable of processing synthetic input data for injection studies in the same manner as measured input data. |
| Test that reply of synthetic input data can be executed and will produce an output file consistent with the measured object data for the same source. |
| GR-10-OUT |
| The replay tool shall provide output (results) in N42.42 -2012 format with device-type appropriate detail. |
| Test that all output files produced by replaying the set of validation input files are valid N42.42 files with detail as required in the detailed level requirements for the device type. |
| GR-11-OUT |
| The replay tool shall provide, when device-type appropriate, traceability (achieved either by inclusion or by cross-reference or pointer) to raw data, ancillary data and analysis parameters in the output (results) file. |
| Verify that all output files produced by replaying the set of verification input files include (when required) raw data, ancillary data, and analysis parameters via cross-reference or inclusion |
| GR-12-OUT |
| The replay tool shall provide version information for the firmware in the output file. |
| Verify that all output files produced by replaying the set of verification input files include (when required) the appropriately tagged version information for the firmware. |
| GR-13-OUT |
| The replay tool shall provide hardware identification in the output (results) file. Hardware identification shall be via the appropriate XML tag from the N42.42 (2012) standard [1]. |
| Check that all output files produced by replaying the set of verification input files include (when required) the appropriately tagged hardware identification. |
| GR-14-OUT |
| Software version for the recognition algorithm used in the replay tool shall appear in the output (results) file for all input files processed by the replay tool. |
| Check that the software version for the replay tool’s detection algorithm is appropriately tagged and listed in all output files produced by replaying the input validation files. |
| GR-15-OUT |
| Software version for the recognition algorithm used in the detection system shall appear in the output (results) file for all output files produced by the in-situ recognition algorithm. |
| Check that the software version for the (in-situ) recognition algorithm is appropriately tagged and listed in all output files produced by the in-situ recognition algorithm. |
DLR-SRPM-1-INP
Input files shall include the required fields, either in native file format or in compliant ANSI n42.42-2012 format.
• Portal Manufacturer
• Instrument ID
• Detector Type and Status (e.g. “operational”, “OK”) for each detector
• Occupancy index
• Station ID
• Conveyance speed
• Calibration Type, model, and coefficients
• For each detector and time slice:
· Start Time, Sample Real Time, and Live Time
· Source Type (background, item, ...)
· Spectrum or count data for each detector linked to calibration data
· Occupancy presence code
• Any pre-processed data as it would be provided to the in-situ algorithm.
| GR-10-OUT |
| Replay tool output (results) files shall include the following fields in compliant ANSI n42.42-2012 format. |
• Input data as above in DLR-SRPM-1-INP
• Algorithm Version and Release Date
• Algorithm Configuration
• Analysis start time and end time
• Background Spectrum
• Referral information – category, confidence value, identity
• Spectrum peak analysis results – peak energy value, peak net area value, peak net area uncertainty value (if available) Test that all output files produced by replaying the set of validation input files are valid N42.42 files with detail
| GR-17-INP |
| Model name and serial number for the radiation detection system shall appear in the input file, regardless of the format (native or ANSI N42.42-2012) used. |
| Check that the files produced by the device and used as input to the replay tool include the appropriately tagged model name and serial number for the detection device. Native format files may be checked by verifying the model name and serial number is present in the translated data files. |
| GR-18-INP |
| Hardware identifier and firmware (when applicable) version(s) for the detection system as specified by the device-type requirements in Table 2, should appear in the input file when possible. |
Firmware includes any firmware used in the detection system to produce the input file.
Check that the files produced by the device and used as input to the replay tool include the appropriately tagged hardware identifiers and firmware version(s) for the detection system.
Check that these data are present in native format files after translation.
| GR-19-INP |
| Background and foreground data shall appear in the input file from the detection system according to the requirements listed in Table 2. |
| Check that the input files produced by the detection system and used as input to the replay tool include (when required) the appropriately tagged background and foreground data. |
| GR-20-BAT |
| The replay tool shall provide a facility for batch processing of a minimum of 100,000 input files without further human intervention. The batch facility should be callable from a command-line interface. |
The batch facility may be a wrapper around a replay tool program that processes a single file, or the primary replay tool facility may have this capability.
Check that a batch processing capability exists.
Check that at least 5,000 files can be processed without further human interaction after launch of the replay tool batch facility.
Check that parameters as designated in the tuning of the replay tool are listed in the output files when the files are processed via batch processing.
Check that the replay tool batch facility is callable from a command-line interface and that output during command-line operation does not overwrite or interfere with file i/o or other replay tool operations.
| GR-21-BAT |
| During batch processing, processing of valid input files shall continue without interruption regardless of the presence of any errant, invalid, incomplete, rejected, or otherwise non-replayable input files. Non-replayable input files shall be skipped by the replay tool and processing of valid input files continue. |
Verify that errant, invalid, incomplete, or otherwise rejected files are skipped by the replay tool and processing of valid input files continues when the errant files are inserted in the beginning, the middle, and the end of the batch of files.
| GR-22-BAT |
| Batch processing in the replay tool shall be subject to all requirements regarding singly-replayed files. |
| Check that each file processed via batch mode corresponds to a uniquely labelled output file. |
Verify that batch processing of input files produces identical output, with the exception of time stamp, regardless of the order in which the files are processed.
| GR-23-BAT |
| Designation of the input files for batch processing shall be either via a replay input directory under which all files will be processed, or via a file of input file names and path specifications to allow processing of files in a path other than that of the replay tool. One or both methods may be implemented to satisfy this requirement. |
| Check that file names can be input to the command-line processing facility via a file of file names, including path, or by dropping the files into the input directory. |
| GR-24-BAT |
| Designation of analysis parameters for batch processing shall be via a file of parameters (see GR-3-REP) either specified at the start of the batch process, or introduced into the replay input directory. |
| Check that analysis parameters can be set by designating the path and filename of a parameter file, or by copying the parameter file into the replay input directory. |
| GR-25-BAT |
| The batch processing facility shall provide a human-readable tabulated summary file from batch processing of files. |
| Check that the batch facility produces a file of tabulated data indicating correctly the number of files from a batch that are processed, skipped, rejected or otherwise not processed, files with errors that are nonetheless processed, and the files that are alarming. |
Check that the file of batch output is human-readable and independent of the order in which the files are processed.
| GR-26-BAT |
| The batch facility shall flag the output of any errant file that can be processed by a unique identifier. |
| GR-27-BAT |
| The batch facility shall provide tabulated data from batch that indicates, for each batch, the number of files from the batch that are processed. |
| GR-28-BAT |
| The batch facility shall provide tabulated data from batch that indicates, for each batch, the number of files from the batch that cannot be processed. |
| GR-29-BAT |
| The batch facility shall provide tabulated data from batch that indicates, for each batch, the number of files that are errant but can be processed. |
| GR-30-BAT |
| The batch facility shall provide tabulated data from batch that indicates, for each batch, the number of alarming and non-alarming files. |
| GR-31-BAT |
| The batch facility shall generate a separate file of the names of non-replayable files from each batch formatted as one filename per line. |
| Insert non-replayable files into a batch processing list in the beginning, middle, and end of the list. Ensure in each instance that all non replayable file names appear in the separate output file’s list of non-replayable files and names of replayed files do not appear in this separate output file’s list. |
| GR-32-COP |
| The replay tool shall be compatible with and provide installation mechanisms for standard-issue x86-based commercial computers running generally available standard-issue operating systems. |
| Check that the replay tool can be installed and run on the designated platforms, and that results from each run of the validation files are identical |
| GR-33-COP |
| The replay installation package shall be complete, including all packages, libraries, third-party software, and licensing required to install and run the platform specific version of the replay tool. |
| Ensure that the installation package includes all required software for correct installation of the replay tool according to installation instructions |
| GR-34-COP |
| Updated versions of the replay tool shall be backward compatible with input files generated by earlier versions of the radiation detection system and processed by earlier versions of the replay tool |
| Verify that the updated replay tool includes a change in the version number as written to the output file. |
Test that subsequent versions of the replay tool are able to replay all files from the validation and verification data sets from previous versions.
Test that subsequent versions of the replay tool indicate the hardware and firmware version, with appropriate tag.
Deliverables
| GR-37-DOC |
| The replay tool shall include a complete manual for installation, operation and troubleshooting. |
Check that the manual is complete and comprehensive
| GR-38-DOC |
| The replay tool documentation shall include the file input and output architecture description. This includes complete description of the data file formats and content. |
| Check that the documentation is delivered, complete, and matches collected data |
File details come from the government source that posted it. Updated .