Attachment F - Testing Guidelines.pdf

Attachment F: Testing Guidelines

CHANGE REQUEST TITLE:

A/GIS/IPS/PRV

Question

Document/Section Number or

Explanation

1. Will this change result in the collection of personally identifiable information (PII), and if so, from what types of individuals (members of the public, foreign nationals, U.S.

Government workers, contractors, legal permanent residents)? For a definition of PII, go to http://ips.state.gov/index.cfm?fuseaction=public.display&s hortcut=4PKY

Yes No N/A

2. Will this change result in the conversion of paper records

(containing PII) to computer-based records?

3. Does this change combine information from two or more sources such that formerly anonymous records can now be used to distinguish an individual or will result in records that are of much higher sensitivity, e.g., personal health or financial facts as opposed to just contact information, e.g., name, e-mail address, phone number, etc.?

4. Will this change adopt a technology that significantly alters how PII is managed, e.g., relational databases, web services, document images, etc.?

5. Will personnel at other U.S. Government agencies have online access to the system/application?

DS/SI/CS

Document/Section Number or

Explanation

6. What category of product does this change request support?

GOTS COTS Customized Hardware

7. Is this request part of a certification effort?

8. Does this proposed change comply with all DoS policies regarding information, computer and network security

(such as 12 FAM 600)?

9. Does any component of this proposed change require downloading files from sources outside of OpenNet or

ClassNet during installation or updating?

http://ips.state.gov/index.cfm?fuseaction=public.display&shortcut=4PKY http://ips.state.gov/index.cfm?fuseaction=public.display&shortcut=4PKY

10. List the country of origin, vendor and manufacturer of all products used as part of this request.

11. Describe all deviations from the mandatory DS security configuration settings (e.g., OS or application-specific) configuration settings.

12. List any waivers authorizing deviations from the standard

OS or Application configuration settings.

13. Was any component of this proposed change developed by anyone other than cleared Americans?

14. What mechanisms will be used for identification and authentication?

15. Describe all network traffic associated with this request that will traverse non-DoS networks.

FSI/SAIT/PDD

Document/Section Number or

Explanation

16. How does the change impact training?

17. Does it require a modification to existing classes?

18. Does it require developing a new class?

19. What is the audience?

20. How many people does the change affect?

21. Is it DoS wide?

22. Bureau?

23. Division?

24. Only a few specific people?

IRM/IA

Document/Section Number or

Explanation

25. Is the product IA or IA -Enabled?

http://irm.m.state.sbu/sites/ia/Pages/Default.aspx

26. Is this product compliant with 5 FAM 915.15-3 and

NSTISSP #11?

27. Is a Deferred Compliance Authorization (DCA) Required?

28. Has a request for DCA been initiated by a memo (with justification to CISO)?

http://irm.m.state.sbu/sites/ia/Pages/Default.aspx

29. Should this product be reported on ITAB?

http://itab.irm.state.gov/Deciding%20What%20to%20Rep ort%20-%20Final.pdf

30. Does this product represent significant modifications to an information system?

http://irm.m.state.sbu/sites/ia/CAToolKit/Webpages/Signif icant_change.html

IRM/SMART

31. Does this change affect the overall performance (i.e., end user experience) of an OpenNet or ClassNet workstation?

32. Does this change affect the SMART client configuration and/or functionality?

33. If so, has the SMART client been tested with this change?

34. Does this change involve sending or receiving emails or cables via SMART?

35. If so, has this change been tested with SMART?

36. Does this change affect SharePoint or MS Communicator?

37. If so, has this change been tested with the SMART functionality that relies on SharePoint or MS

Communicator?

38. Does this change affect any SMART server configurations

(e.g., BizTalk, SQL Server, Oracle, etc.)?

39. If so, has this change been tested with the SMART core servers?

Functional Bureau

Document/Section Number or

Explanation

40. Is this product (when deployed) to be of limited scope or is it going to be required for use for every bureau/post?

41. Who will install it, who provides technical support?

42. Is there documentation and installation instructions beyond those provide by the vendor?

43. Is it a service to be centrally managed?

44. What training is available for users as well as techs?

45. Is this a pilot?

46. If yes, how long, where, and how many nodes?

47. Will results of the pilot be presented to the IT CCB prior to http://itab.irm.state.gov/Deciding%20What%20to%20Report%20-%20Final.pdf http://itab.irm.state.gov/Deciding%20What%20to%20Report%20-%20Final.pdf http://irm.m.state.sbu/sites/ia/CAToolKit/Webpages/Significant_change.html http://irm.m.state.sbu/sites/ia/CAToolKit/Webpages/Significant_change.html a request for full deployment?

48. Is this is an appliance

49. Is there evidence of review of the product beyond provision of vendor provided security and installation documents?

50. How do I make it work within the State Department configuration and guidelines?

51. Is it required to upgrade workstations or servers and if so within what time frame?

Yes No

IRM/BPC/CST

Document/Section Number or

Explanation

52. As required by Federal Civil Rights, IT, & Acquisition law

(29 USC 794d & 36 CFR 1194), as well as Department IT policy in the FAM/FAH, do the COTS/customized E&IT products/services associated with this AR/CR/SAR comply with/conform to Section 508?

53. Is each E&IT product/service accessible to and usable by all DoS users with disabilities or accessible to and usable by the public users with disabilities where DoS provides such products/services to the public (regardless of whether or not such DoS/public users with disabilities exist currently)?

54. Well in advance to submitting the request to the DoS IT

CCB; has the originator of the AR/CR/SAR or any related vendors/developers/project or product managers consulted/coordinated/agreed to an action plan for meeting all necessary requirements with the CST/IMPACT

Technical Reviewer?

55. Well in advance to submitting the request to the DoS IT

CCB, has a VPAT been submitted for each COTS E&IT product/service (e.g., software/web/hardware)?

56. For each product/service in the VPAT, has the vendor/developer justify/ground sufficiently the column-2 responses/claims of conformance via one or more acceptable means (e.g., describing briefly in column 3 or separately code used to implement the given line-item/requirement, Providing internal or IV&V 508 testing results and tools used, including standard adaptive technology, as indicated by 36 CFR 1194.31)?

57. Well in advance to submitting the request to the DoS IT

CCB, has a GPAT been submitted for each customized/developed E&IT product/service (e.g., software/web/hardware)?

58. For each product/service in the GPAT, has the DoS IT project manager or developer justify/ground sufficiently the column-2 responses/claims of conformance via one or more acceptable means (e.g., describing briefly in column

3 or separately code used to implement the given line-item/requirement, Providing IV&V 508 testing results and tools used, including standard adaptive technology, as indicated by 36 CFR 1194.31)?

59. Has each VPAT or GPAT been fully and properly completed by the drafter and reviewed and determined acceptable by the CST/IMPACT Technical Reviewer?

60. Has the submitter of the AR/CR/SAR addressed sufficiently its impact on foreign operations?

IRM/BPC/RG

Document/Section Number or

Explanation

61. Are the policies in 5 FAM 600 (Information Technology

Systems), specifically 5 FAM 620 (Project Management) being followed?

62. Are the policies in 5 FAM 800 (Information Systems

Management), specifically 5 FAM 860 (Hardware and

Software Maintenance) and 5 FAM 870 (Networks) being followed?

63. Are the procedures in 5 FAH-5 (Information Technology

Systems Handbook) being followed?

IRM/OPS/ENM

Document/Section Number or

Explanation

64. What is your plan for version control of hardware and software both primary and sub components?

65. What is your approach for Patch Management for both primary and sub components?

66. How long is this product estimated to be on the DoS

Baseline (e.g. life cycle of 1, 3, 5, 20 years)?

67. What is your approach for maintenance of the core applications during the lifecycle for both primary and sub components?

68. Define the plan for compliance with major areas of the

Department’s IT Baseline (ex: Operating System)

69. What is the plan for license management for both primary and sub components?

70. What is the planned deployment methodology?

71. Provide a list of all hardware and software both primary and sub components and a configuration drawing.

72. Identify network impact and methodology used to make the determination. Provide test results if available. (i.e., X bits per second for a duration of X minutes for day by X number of users per day)

73. Is this system utilizing the DoS Active Directory for access authentication? If not, why?

74. Is the software used compatible with IPv6?

IRM/OPS/ITI/SI/IIB

Document/Section Number or

Explanation

75. Was DoS approved Antivirus software included in testing?

Yes – Please list version(s) used.

No – Please provide explanation, including what other

Antivirus solution is being proposed.

76. Is the CR compatible with currently approved Antivirus software?

Yes

No – Please provide explanation.

77. Are there any special Antivirus requirements related to this

CR?

Yes – Please provide explanation.

No

78. Was DoS approved PKI software included in testing?

Yes – Please list version(s) used.

No – Please provide explanation.

70. Does this CR include any encryption services such password stores, data encryption, or communication link security (SSL, TLS, 3DES, AES)?

Yes – Please note, per FISMA, ALL modules implementing cryptographic services must be FIPS 140-2 certified. Please provide the relevant certificate number, available at:

http://csrc.nist.gov/groups/STM/cmvp/documents/140-

1/140val-all.htm. Requests to implement non-FIPS 140-2 certified cryptographic services MUST be reviewed and approved in writing by DS and IA.

71. Does the application require the use of PKI or other managed cryptographic services?

Yes – Please contact PKI Program in

IRM/OPS/ITI/SI/IIB.

72. Does this system require any interfaces to the legacy applications CLASS-E, CAPPS, FARADS, CFMS, BFMS, RMS, or BRMS?

Yes – Please contact Mainframe Security Program in

IRM/OPS/ITI/SI/IIB.

http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm

73. Is your application Internet accessible AND does it contain anything other than non-sensitive, public read only data?

No – proceed to question 74

Yes, please answer the applicable sub-questions below.

a. Have you consulted with IA to conduct an Electronic

Risk and Requirements Assessment (e-RA)?

Yes, answer questions 73b and 73c.

No – STOP will be issued until results of consultation with IA can be reported.

b. What assurance level was reported out by the e-RA tool?

c. What authentication technology was selected and what, if any, compensating controls were included?

74. Does your application require any type of electronic signature, digital or otherwise?

Yes – see 5 FAM 140 for compliance requirements.

IRM/OPS/MSO

Document/Section Number or

Explanation

75. Does the application/system communicate outside of

OpenNet/ClassNet?

76. Does the application/system receive communications from outside OpenNet/ClassNet?

77. If the answer to either of the above is 'yes' then

a. Describe point-to-point connectivity requirements, if any.

b. What ports and protocols does the application/system use?

78. Will this application/system require Internet access?

79. Are port requirements configurable?

80. Are application/system communications RFC compliant?

(Reference: http://www.rfc.net/).

81. Are there encryption requirements? If so, please describe.

82. Are there remote management requirements? If so, please describe.

83. Is the application/system

Read-only Interactive?

84. What is the size of the user community?

85. Timeframe to implementation?

86. Does this application/system interface with the existing

DoS telegram system in any way?

87. Will this application/system be installed on any client workstation that is used to process DoS telegrams?

88. Does the application/system require DNS changes?

89. Will this application/system interface with a directory service?

If the answer above is 'yes' then define which one …

90. Will this application/system interact with MS Exchange?

91. Will this application/system interact with SIPRNET?

92. Will this application be required to work with OpenNet

Everywhere (ONE)?

93. If the answer is yes, then has it been tested on OpenNet

Everywhere (ONE) (explain)?

94. Will this application need to be mobile accessible?

If the answer is yes, then has it been tested for mobile accessibility (explain)?

IRM/OPS/SIO

Document/Section Number or

Explanation

95. Have you contacted the owners of business applications that could be affected by the activity associated with this request?

Yes - Additional time for testing applications may be required.

No – Contact application owners.

N/A – No other applications affected. (Ignore the next

2 questions.)

96. Will this request have possible impact on (require changes to or retesting of) PASS applications?

Yes - Contact PASS PMO. PASS PMO may notify

PASS application owners. Additional time for testing

PASS applications may be required.

No - No action required.

97. Will this request have possible impact on (require changes to or retesting of) Payroll applications?

Yes - Notify application owners. Additional time for testing applications may be required.

98. Have any changes that affect the Data Center or ESOC been coordinated with the ESOC staff?

Yes - No action required.

No - Contact ESOC and initiate coordination.

99. Will this CR place additional network load or power requirements on servers already installed in the ESOC?

Yes - Work with ESOC to estimate the increase and ensure that resources are available.

100. Is the application registered in the IT Application Base

(ITAB)?

Yes – Please enter the ITAB system number under

“Document Name / Explanation”

No - Contact ITAB program office to discuss registering this application.

101. Is the mandatory application information in ITAB complete?

Yes

No - Certify application information in ITAB as complete

102. Is this an application, database, or information collection that contains line of business information? (See ITAB system definition guidelines)

Yes - Complete remainder of the IRM/OPS/SIO checklist.

No - The remainder of the IRM/OPS/SIO checklist does not need to be completed.

103. Provide Data Steward name and contact information:

Name:

Organization:

Phone: - -

E-mail: @ state.gov

Location:

104. Describe the data/information contained within the application.

105. Is the application database metadata loaded into the

Department Metadata Repository?

Yes - No action required.

No - Contact IRM/OPS/SIO/API to arrange for loading metadata into the Department Metadata Repository.

106. Is there a logical data model?

Yes - provide logical data model to Data Management.

No - If application is a custom system, provide the physical data model to Data Management.

107. Are there data definitions and business rule definitions?

Yes - provide data definitions and business rules to

Data Management.

No - Provide plan to develop definitions.

108. Does system interface with other systems electronically or is information transferred from system into other systems manually?

Yes - Complete Data Flow section of ITAB.

No. – No action required.

109. Does the system use XML?

Yes - Provide XML schema definitions to Data

Management; use XML guidelines of XML Working

Group.

No – No action required.

http://itab.irm.state.gov/

110. Does the data in this application duplicate data in other applications in the Department or other Agencies?

Yes - Contact other system owners and provide justification for duplication.

111. Are there other potential users of this data, internal or external to the Department, or both (e.g. other organizations that can also benefit from this data)?

Yes - Please identify in “Document Name /

Explanation” or reference document in which these organizations are identified.

RM/BP

Document/Section Number or

Explanation

112. Is the cost comparable to the previous model/version?

113. Is there any difference in the replacement item that will cause the Department to incur additional costs (e.g.

upgrade from MS Office 2003 to MS Office 2007 requires all systems that depend on MS Office to test and potentially modify software, whereas the replacement of the baseline workstation monitor is unlikely to have other costs)?

Yes Describe:

No N/A

114. If this request is for a product (e.g. software package, hardware item), what is the cost per item?

115. If this request is for a specific product, under the scenario for its use as proposed in the documentation, how many will be purchased? _____

a. How often? _____

b. The number purchased depends. Describe what it depends on in the block to the right.

N/A

116. If this request is for a product, did the vendor provide an incentive to implement this product (e.g. no cost implementation for testing)?

If the answer above is YES (the vendor provided an incentive) what due diligence was done to provide access to competitive products? Description:

117. What other costs are associated with implementing this product (e.g. server, installation, recurring maintenance costs)?

Description: _________________________________

Amount: ____________

118. If this request is granted, how many bureaus besides yours will incur a cost due to the implementation described in your documents?

All Most None

119. If this request is for a product, are other bureaus not required, but may want to utilize the item?

120. Have the necessary purchases for this request already been made?

If yes, how much was spent?

121. For bureau managed systems, what eCPIC entry represents the budget of this request? Name: __________________

If no eCPIC entry exists, then…

a. Has the project coordinated with the eGovPMO office?

Yes No

b. Is the system funded?

Yes No N/A

c. From what source? Name or number of fund:

d. Will deployment/installation of this request require any expenditure at bureaus/posts besides yours?

Yes No N/A

e. Will deployment/installation assume existing resources at bureaus/posts (e.g. an available server minimum/maximum bandwidth standards, etc.)?

FOR POST REQUESTS:

122. Is the system funded?

123. From what source? Name or number of fund and or other agencies contributing to cost:

124. Was the subject of the request implemented using equipment made available when GITM replaced equipment?

125. Is this the first year of an annual cost?

126. Was this a fiscal year-end purchase?

IRM/OPS/ITI-GITM

127. Is the product intended to be added to the SOE-D, or installed on a standard member server?

SOE-D Server N/A

128. If so, has it been tested on a current GITM workstation or server image?

Yes No N/A

If yes, please provide the test report as part of your submission.

129. Was such testing coordinated with GITM?

130. Were any products that are already IT CCB-approved considered for providing this capability?

Yes No If so, why were they rejected?