14-RP-0022.pdf

RFQ IFB RFP

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

1. REQUISITION NUMBER PAGE OF

2. CONTRACT NO. 3. AWARD/EFFECTIVE

DATE

4. ORDER NUMBER 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE

DATE

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME b. TELEPHONE NUMBER (No collect calls)

8. OFFER DUE DATE/

LOCAL TIME

9. ISSUED BY

13b. RATING

14. METHOD OF SOLICITATION

CODE

15. DELIVER TO 16. ADMINISTERED BY CODE

18a. PAYMENT WILL BE MADE BY CODEFACILITY

CODE

CODE

TELEPHONE NO.

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN

OFFER

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK

BELOW IS CHECKED

SEE ADDENDUM

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED

29. AWARD OF CONTRACT: REF.

DATE . YOUR OFFER ON SOLICITATION

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

SET FORTH HEREIN, IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR

30b. NAME AND TITLE OF SIGNER (Type or print) 30c. DATE SIGNED

31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

31b. NAME OF CONTRACTING OFFICER (Type or print) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

STANDARD FORM 1449 (REV. 2/2012)

Prescribed by GSA - FAR (48 CFR) 53.212

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND

OFFER

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

ARE ARE NOT ATTACHED

ARE ARE NOT ATTACHED

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA

17a CONTRACTOR/

OFFEROR.

CODE

8 (A)

SIZE STANDARD:

NAICS:

% FOR:SET ASIDE:UNRESTRICTED OR

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SMALL BUSINESS

10. THIS ACQUISITION IS

EDWOSB

SMALL BUSINESS PROGRAM

STANDARD FORM 1449 (REV. 2/2012) BACK

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED:

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

32b. SIGNATURE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32c. DATE

41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER 41c. DATE

42a. RECEIVED BY (Print)

42b. RECEIVED AT (Location)

42c. DATE REC'D (YY/MM/DD) 42d. TOTAL CONTAINERS

40. PAID BY

32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE 32f. TELPHONE NUMBER OF AUTHORZED GOVERNMENT REPRESENTATIVE

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

33. SHIP NUMBER 34. VOUCHER NUMBER 35. AMOUNT VERIFIED

CORRECT FOR

PARTIAL FINAL

37. CHECK NUMBER

38. S/R ACCOUNT NO. 39. S/R VOUCHER NUMBER

36. PAYMENT

COMPLETE PARTIAL FINAL

SCHEDULE Continued

ITEM NO. SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

PAGE 3 OF 98 SB1341-14-RP-0022

Base Period 06/02/2014 to 06/01/2015

NTE NTE

0001 Contractor performance of the requirements set forth in

Section C Statement of Work

PR NUMBER: NB774000-14-01795

Period of Performance: 06/02/2014 to 06/01/2015

1.00 LO

06/02/2014 to 06/01/2015

NTE NTE

0002 Other Direct Cost: Not to exceed $100,000.00

NTE NTE

0003 Travel Cost: Not to exceed $150,000

NTE NTE

0004 Test participant compensation: Not to exceed

$200,000.00

Option Period I

NTE NTE/OPT

0005 Contractor performance of the requirements set forth in

Section C Statement of Work

Period of Performance: 06/02/2015 to 06/01/2016

0006 Other Direct Cost: Not to exceed $100,000.00

0007 Travel Cost: Not to exceed $150,000

0008 Test participant compensation: Not to exceed

$200,000.00

Option Period II

0009 Contractor performance of the requirements set forth in

Section C Statement of Work

Period of Performance: 06/02/2016 to 06/01/2017

0010 Other Direct Cost: Not to exceed $100,000.00 1.00 LO

PAGE 4 OF 98 SB1341-14-RP-0022

0011 Travel Cost: Not to exceed $150,000

0012 Test participant compensation: Not to exceed

$200,000.00

Option period III

0013 Contractor performance of the requirements set forth in

Section C Statement of Work

Period of Performance: 06/02/2017 to 06/01/2018

Option Period III

0014 Other Direct Cost: Not to exceed $100,000.00

0015 Travel Cost: Not to exceed $150,000

0016 Test participant compensation: Not to exceed

$200,000.00

Option Period IV

0017 Contractor performance of the requirements set forth in

Section C Statement of Work

Period of Performance: 06/02/2018 to 06/01/2019

0018 Other Direct Cost: Not to exceed $100,000.00

Option IV

0019 Travel Cost: Not to exceed $150,000

Period of Performance: 06/02/2014 to 06/01/2019

0020 Test participant compensation: Not to exceed

$200,000.00

PAGE 5 OF 98 SB1341-14-RP-0022

Table of Contents

PAGE 6 OF 98 SB1341-14-RP-0022

SECTION B SUPPLIES OR SERVICES AND PRICES/COSTS

B. 1 1352.216-75 MINIMUM AND MAXIMUM CONTRACT AMOUNTS (APR 2010)

B. 2 1352.216-77 CEILING PRICE (APR 2010)

SECTION C DESCRIPTION/SPECIFICATIONS/WORK STATEMENT

C. 1 Statement of Work

SECTION E INSPECTION AND ACCEPTANCE

E. 1 52.246-4 INSPECTION OF SERVICES--FIXED-PRICE (AUG 1996)

SECTION F DELIVERIES OR PERFORMANCE

F. 1 52.242-15 STOP-WORK ORDER (AUG 1989)

F. 2 52.242-17 GOVERNMENT DELAY OF WORK (APR 1984)

F. 3 1352.270-70 PERIOD OF PERFORMANCE (APR 2010)

SECTION G CONTRACT ADMINISTRATION DATA

G. 1 1352.201-70 CONTRACTING OFFICER?s AUTHORITY (APR 2010)

G. 2 1352.201-72 CONTRACTING OFFICER?s REPRESENTATIVE (C0R) (APR 2010)

G. 3 1352.216-76 PLACEMENT OF ORDERS (APR 2010)

G. 4 1352.245-70 GOVERNMENT FURNISHED PROPERTY (APR 2010)

G. 5 IDIQ Contract Information

SECTION H SPECIAL CONTRACT REQUIREMENTS

H. 1 1352.208-70 RESTRICTIONS ON PRINTING AND DUPLICATING (APR 2010)

H. 2 1352.209-73 COMPLIANCE WITH THE LAWS (APR 2010)

H. 3 1352.209-74 ORGANIZATIONAL CONFLICT OF INTEREST (APR 2010)

H. 4 1352.231-71 DUPLICATION OF EFFORT (APR 2010)

H. 5 1352.216-74 TASK ORDERS (APR 2010)

H. 6 1352.227-70 RIGHTS IN DATA, ASSIGNMENT OF COPYRIGHT (APR 2010)

H. 7 1352.228-70 INSURANCE COVERAGE (APR 2010)

H. 8 1352.228-72 DEDUCTIBLES UNDER REQUIRED INSURANCE COVERAGE- FIXED PRICE (APR 2010)

H. 9 1352.233-70 AGENCY PROTESTS (APR 2010)

H. 10 1352.235-70 PROTECTION OF HUMAN SUBJECTS (APR 2010)

H. 11 1352.235-71 PROTECTION OF HUMAN SUBJECTS -- EXEMPTION (APR 2010)

H. 12 1352.235-72 PROTECTION OF HUMAN SUBJECTS - INSTITUTIONAL APPROVAL (APR 2010)

H. 13 1352.235-73 RESEARCH INVOLVING HUMAN SUBJECTS - AFTER INITIAL CONTRACT AWARD (APR 2010).. 34

H. 14 1352.237-71 SECURITY PROCESSING REQUIREMENTS - LOW RISK CONTRACTS (APR 2010)

H. 15 1352.237-75 KEY PERSONNEL (APR 2010)

H. 16 1352.239-72 SECURITY REQUIREMENTS FOR INFORMATION TECHNOLOGY RESOURCES (APR 2010)

H. 17 NIST LOCAL-07 COMPUTER SECURITY POLICY

H. 18 NIST LOCAL-40 BILLING INSTRUCTIONS FOR DELIVERY/TASK ORDERS

SECTION I CONTRACT CLAUSES

I. 1 52.202-1 DEFINITIONS (NOV 2013)

I. 2 52.203-3 GRATUITIES (APR 1984)

I. 3 52.203-5 COVENANT AGAINST CONTINGENT FEES (APR 1984)

I. 4 52.203-6 I RESTRICTIONS ON SUBCONTRACTOR SALES TO THE GOVERNMENT (SEP 2006)-- ALTERNATE I

(OCT 1995)

I. 5 52.203-7 ANTI-KICKBACK PROCEDURES (OCT 2010)

I. 6 52.203-8 CANCELLATION, RESCISSION, AND RECOVERY OF FUNDS FOR ILLEGAL OR IMPROPER ACTIVITY

(JAN 1997)

PAGE 7 OF 98 SB1341-14-RP-0022

I. 7 52.203-10 PRICE OR FEE ADJUSTMENT FOR ILLEGAL OR IMPROPER ACTIVITY (JAN 1997)

I. 8 52.203-12 LIMITATION ON PAYMENTS TO INFLUENCE CERTAIN FEDERAL TRANSACTIONS (OCT 2010)

I. 9 52.204-4 PRINTED OR COPIED DOUBLE-SIDED ON POSTCONSUMER FIBER CONTENT PAPER (MAY 2011)

I. 10 52.204-7 SYSTEM FOR AWARD MANAGEMENT (JUL 2013)

I. 11 52.204-9 PERSONAL IDENTITY VERIFICATION OF CONTRACTOR PERSONNEL (JAN 2011)

I. 12 52.204-10 REPORTING EXECUTIVE COMPENSATION AND FIRST-TIER SUBCONTRACT AWARDS (JULY 2013). 41

I. 13 52.209-6 PROTECTING THE GOVERNMENT` INTEREST WHEN SUBCONTRACTING WITH CONTRACTORS DEBARRED, SUSPENDED, OR PROPOSED FOR DEBARMENT (AUG 2013)

I. 14 52.209-7 INFORMATION REGARDING RESPONSIBILITY MATTERS (JULY 2013)

I. 15 52.212-4 CONTRACT TERMS AND CONDITIONS--COMMERCIAL ITEMS (SEPT 2013)

I. 16 52.215-2 AUDIT AND RECORDS--NEGOTIATION (OCT 2010)

I. 17 52.215-8 ORDER OF PRECEDENCE--UNIFORM CONTRACT FORMAT (OCT 1997)

I. 18 52.222-50 COMBATING TRAFFICKING IN PERSONS (FEB 2009)

I. 19 52.223-5 POLLUTION PREVENTION AND RIGHT-TO-KNOW INFORMATION (MAY 2011)

I. 20 52.223-6 DRUG-FREE WORKPLACE (MAY 2001)

I. 21 52.224-1 PRIVACY ACT NOTIFICATION (APR 1984)

I. 22 52.224-2 PRIVACY ACT (APR 1984)

I. 23 52.227-1 AUTHORIZATION AND CONSENT (DEC 2007)

I. 24 52.228-5 INSURANCE--WORK ON A GOVERNMENT INSTALLATION (JAN 1997)

I. 25 52.229-3 FEDERAL, STATE, AND LOCAL TAXES (FEB 2013)

I. 26 52.232-1 PAYMENTS (APR 1984)

I. 27 52.232-8 DISCOUNTS FOR PROMPT PAYMENT (FEB 2002)

I. 28 52.232-11 EXTRAS (APR 1984)

I. 29 52.232-17 INTEREST (OCT 2010)

I. 30 52.233-1 DISPUTES (JUL 2002)

I. 31 52.233-3 PROTEST AFTER AWARD (AUG 1996)

I. 32 52.233-4 APPLICABLE LAW FOR BREACH OF CONTRACT CLAIM (OCT 2004)

I. 33 52.237-2 PROTECTION OF GOVERNMENT BUILDINGS, EQUIPMENT, AND VEGETATION (APR 1984)

I. 34 52.242-13 BANKRUPTCY (JUL 1995)

I. 35 52.243-1 I CHANGES--FIXED-PRICE (AUG 1987)--ALTERNATE I (APR 1984)

I. 36 52.244-2 SUBCONTRACTS (OCT 2010)

I. 37 52.244-5 COMPETITION IN SUBCONTRACTING (DEC 1996)

I. 38 52.246-23 LIMITATION OF LIABILITY (FEB 1997)

I. 39 52.246-25 LIMITATION OF LIABILITY--SERVICES (FEB 1997)

I. 40 52.249-1 TERMINATION FOR CONVENIENCE OF THE GOVERNMENT (FIXED-PRICE) (SHORT FORM) (APR

1984)

I. 41 52.253-1 COMPUTER GENERATED FORMS (JAN 1991)

I. 42 52.212-5 CONTRACT TERMS AND CONDITIONS REQUIRED TO IMPLEMENT STATUTES OR EXECUTIVE ORDERS--

COMMERCIAL ITEMS (JAN 2014)

I. 43 52.216-18 ORDERING (OCT 1995)

I. 44 52.216-19 ORDER LIMITATIONS (OCT 1995)

I. 45 52.216-22 INDEFINITE QUANTITY (OCT 1995)

I. 46 52.217-8 OPTION TO EXTEND SERVICES (NOV 1999)

I. 47 52.217-9 OPTION TO EXTEND THE TERM OF THE CONTRACT (MAR 2000)

I. 48 52.227-11 PATENT RIGHTS--OWNERSHIP BY THE CONTRACTOR (DEC 2007)

I. 49 52.227-14 RIGHTS IN DATA--GENERAL (DEC 2007)

I. 50 52.227-17 RIGHTS IN DATA--SPECIAL WORKS (DEC 2007)

I. 51 52.227-18 RIGHTS IN DATA--EXISTING WORKS (DEC 2007)

I. 52 52.246-6 INSPECTION--TIME-AND-MATERIAL AND LABOR-HOUR (MAR 2001)

I. 53 52.252-2 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)

PAGE 8 OF 98 SB1341-14-RP-0022

SECTION J LIST OF ATTACHMENTS

J. 1 Format for submitting questions

J. 2 Past performance questionaire template

J. 3 Task Order 1

SECTION K REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS

K. 1 52.204-8 ANNUAL REPRESENTATIONS AND CERTIFICATIONS (JAN 2014)

K. 2 52.212-3 OFFEROR REPRESENTATIONS AND CERTIFICATIONS--COMMERCIAL ITEMS (NOV 2013)

SECTION L INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS

L. 1 52.212-1 INSTRUCTIONS TO OFFERORS--COMMERCIAL ITEMS (JULY 2013)

L. 2 52.216-1 TYPE OF CONTRACT (APR 1984)

L. 3 52.233-2 SERVICE OF PROTEST (SEP 2006)

L. 4 Inquiries

L. 5 Proposal and preparation submission instructions

SECTION M EVALUATION FACTORS FOR AWARD

M. 1 52.217-5 EVALUATION OF OPTIONS (JUL 1990)

M. 2 IDIQ Evaluation Criteria

PAGE 9 OF 98 SB1341-14-RP-0022

SECTION B

SUPPLIES OR SERVICES AND PRICES/COSTS

B. 1 1352.216-75 MINIMUM AND MAXIMUM CONTRACT AMOUNTS (APR 2010)

During the term of the contract, the Government shall place orders totaling a minimum of 20,000.00 .

The amount of all orders shall not exceed 5,500,000.00 .

(End of clause)

B. 2 1352.216-77 CEILING PRICE (APR 2010)

The ceiling price of this contract is $5,500,000.00 . The contractor shall not make expenditures nor incur obligations in the performance of this contract which exceed the ceiling price specified herein, except at the contractor?s own risk.

PAGE 10 OF 98 SB1341-14-RP-0022

SECTION C

DESCRIPTION/SPECIFICATIONS/WORK STATEMENT

C. 1 Statement of Work

Statement Of Work

Usability and Security IDIQ

I. Background Information

The National Institute of Standards and Technology’s (NIST) Visualization and Usability Group (VUG) performs research in usability and human factors of security and biometric systems. Biometrics are often viewed as a subset of security. Usability is an often overlooked but critical component of cyber security. There is an inherent tradeoff between cyber security and usability. Computers can be theoretically secure but so unusable that they do not improve the security because the users are forced to perform in less secure ways. The opposite is true as well; systems that are usable and not secure are eventually rendered useless due to worms, viruses, and botnets. Usability principles must be incorporated into systems in order to ensure that it is easy for users to do the right thing and hard for them to do the wrong thing.

To fully understand the user it is essential to embrace the considerable social science literature on usability and computer systems. This information includes not only the literature on crime and misbehavior (relevant to the problem’s security aspects) but also the literature on risk, perception, organizational culture, and behavioral decision-making (relevant both to security and usability). By broadening the perspective, the Government should be able to build usability into our security products and processes, spanning the life of a system from conception through design, implementation, and evolution. Good usability, like good security, takes into account the changing nature of the threat environment, changing business goals, and differences in user needs, skills, attitudes, and experience. Moreover, usable security should be tailored to organizational priorities and needs. But one must begin by understanding the user and observing the user. Usability is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” according to ISO 9241-11. Usability is more than a well designed user interface.

The U.S. government has also recognized the need for usable security by initiating programs such as the Comprehensive National Cyber Security Initiative (CNCI) which include usable security research in Initiative 9 which defines and develops enduring “leap-ahead” technology, strategies, and programs. The National Academy of Science convened a workshop in July 2009, sponsored by NSF and NIST, to “identify promising research directions to advance usability, security, and privacy.” The NIST usability team is supporting the Comprehensive Cyber Security Initiative the National Cyber Security Center of Excellence, as well as security programs within the Department of Homeland Security, the Federal Bureau of Investigation, and the Department of Defense to balance the needs of security and usability.

The NIST usability team’s goal is to provide policy makers and systems implementers with usability data to make better informed decisions about security policies and system implementations. Policy makers and implementers currently have security data based on both empirical data and theoretical data but little or no empirical usability data is

PAGE 11 OF 98 SB1341-14-RP-0022

available to support these decisions. The NIST usability team is currently in the process of developing usability tests and protocols to collect this usability data to balance the security data.

II. Scope of Work

The services detailed in this Statement of Work (SOW) shall be provided through an Indefinite-Delivery, Indefinite-Quantity (IDIQ) contract. The scope of this requirement is for contractor provision of all labor, materials, equipment, project oversight, administration and technical expertise required to perform usability testing tasks as defined in individual task orders. Usability testing is defined as: evaluation that involves representative users performing specific tasks with the system to enable the measurement of efficiency, effectiveness, and/or user satisfaction. The purpose of this SOW is to identify tasks that are necessary in the collection of usability data. Under this IDIQ contract, the contractor shall provide usability testing services. The contractor shall be responsible for performing any or all of the following tasks, which are detailed further in Section III of this SOW, under any given task order:

• Recruiting and screening of test participants;

• Providing usability testing facilities:

• Developing test materials and test instrumentation;

• Carrying out or assisting in carrying out test protocols;

• Proctoring of surveys;

• Facilitating interviews and contextual analysis;

• Analyzing data;

• Compensating test participants

• Project managing

III. Specific Tasks

The Contractor shall provide all labor, materials, equipment, project oversight, administration and technical execution of each task order under this contract. The Contractor shall be responsible for maintaining accurate records of project activities under each individual task order. Each task order under this contract will be based on a particular field of interest (such as user perception of biometrics or affordance of biometric devices and symbols, or memorability of passwords in cyber-security, or speech recognition, or health records) within the Usability Studies field. The contractor shall be responsible for any or all of the following specific tasks under a given task order.

Each task order will specify which of the following tasks the contractor shall be responsible for. The following descriptions are summaries of the tasks, and the exact descriptions of the specific tasks to be performed by the contractor will be detailed in each task order.

1. Recruiting of Test Participants: The contractor shall recruit and screen test participants in accordance with the requirements provided by the Contracting

PAGE 12 OF 98 SB1341-14-RP-0022

Officer’s Representative (COR) in the test protocol. The COR will provide the test protocol to the contractor at or near the beginning of each applicable task order. The test protocol will identify the number of participants to be recruited and the demographic characteristics such as age, education, ethnicity, computer experience, language proficiency including languages such as Egyptian Arabic and Iraqi Arabic, etc. The contractor shall manage the scheduling of the participants to participate in the study.

2. Usability Testing Facilities: The contractor shall provide a state of the art (codified below) usability testing laboratory at its facilities. This laboratory shall include the ability to record both audio and video of the usability tests. The facility shall have the ability to record multiple video recordings simultaneously.

The facility shall have a one way mirror for usability researchers to observe the testing as well as the ability to provide a video feed of the testing to a remote observation room. The facility shall have the ability for synchronized screen capture and video capture to allow for observing test participant’s expressions and the screen. The facility shall come equipped with adjustable cameras. The screen capture capability shall be delivered to the NIST usability team at the end of the testing.

Min lab requirements:

State of the Art must include:

• ability to record both audio and video of the usability tests

• ability to remotely adjust camera positions and audio volume

• ability to record multiple video recordings simultaneously

• have a one way mirror for usability researchers to observe the testing in separate observation room

• ability to provide a video feed of the testing to a remote observation room

• ability for synchronized screen capture and video capture to allow for observing test participant’s expressions and the screen

• adjustable lighting

3. Developing Test Materials and Test Instrumentation: The contractor shall develop and/or provide test materials and test instrumentation, as specified in the test protocols. The test materials and test instrumentation provided by the contractor are not for purchase by NIST and will be dedicated solely to the individual task being performed throughout the period of performance of said task. This may include both software and hardware components, for example usability software testing tools such as Morae, Camtasia, or Noldus. Hardware instrumentation may include but is not limited to eye-trackers and other physical measurement devices such as calipers, protractors, or height charts.

PAGE 13 OF 98 SB1341-14-RP-0022

Equipment and Hardware environment requirements

• Current hardware platforms to include desktop and mobile including tablets and smartphones

• Current operating systems (windows and apple) for platforms above (versions of OS should be within 2 of latest release).

• Data to be provided in excel readable format

• Audio in MP3

• Photos in JPEG

• Video in Quicktime, Windows Movie or MP4

4. Carrying out or assisting in carrying out Test Protocols: The contractor shall carry out or assist the NIST Principal Investigator in carrying out the test protocol as provided by the COTR. The contractor shall assist the NIST Principal Investigator in setting up the test in the usability lab. This may include but is not limited to setting up the audio and video recordings and any other data collection mechanisms specified in the test protocol. The NIST Principal Investigator will provide any required testing materials to include software or hardware except those identified in task 2 and 3. The contractor shall assist in greeting test participants and attend to participants as they are waiting before and after the testing. The contractor shall assist the NIST researcher with the tests as specified in the test protocol, which will be provided by the COR at or near the beginning of each applicable task order. (See example protocol)

5. Proctoring of surveys: The contractor shall assist the NIST Investigator in developing the survey instrument and shall distribute surveys in accordance with the requirements in the test protocol. The contractor shall identify appropriate survey participants that meet the criteria specified in the test protocol and provide the survey instrument.

6. Facilitating interviews and contextual analysis: The contractor shall assist NIST Principal Investigator in performing interviews and contextual analysis in accordance with the test protocol. Contextual analysis is defined here as an interviewing process which includes an interpretive field research technique that gathers detailed information about work practice in the workplace while people work through observation and discussion of on-going work based on a model of apprenticeship to the customer. Key concepts include understanding the customer needs in the context of their work, working with customers as partners in the inquiry, uncovering the meaning and implications of customer action and language and listening and probing with clear objectives. The contractor shall

PAGE 14 OF 98 SB1341-14-RP-0022

observe the interviews and take notes, transcribe the audio recordings, and assist the NIST researcher during the interview as specified in the test protocol.

7. Analyzing data: The contractor shall take the collected or raw data from the usability tests and analyze the data as specified in the test protocol. This analysis may include but is not limited to statistical analysis of quantitative data, coding of qualitative data, linguistic analysis, and analysis of survey data.

8. Compensation of test participants: The contractor shall compensate test participants in accordance with the test protocol as provided by the COR. For each task order, the payment amount for each participant may vary depending on the nature of the work and will be defined in each subsequent task. The test protocol will specify this amount.

9. Project Management: The contractor shall manage the quality and timeliness of all tasks and deliverables detailed in individual task orders. This may include correspondence with the NIST COR on a periodic basis to be specified in individual task orders, informing the COR on the progress of the work being performed and any pertinent issues.

Minimum Personnel Requirements

• To observe usability testing or follow a script (for example for recruiting) – 2 years min.

• For facilitation of a usability test a min of 3 to 5 years.

• For any design of a usability test or a survey or anything else a minimum of 5 years.

• For program management – a min of 5 years.

IV. Deliverables and Reports

The Contractor shall provide the required deliverables and reports in accordance with the Statement of Work of each applicable task order under this contract.

Each task order under this contract will be based on a particular field of interest within the Usability Studies field. The contractor shall be responsible for the deliverables that are detailed in each individual task order. The following are samples of some deliverables that the contractor may be responsible for in a given task order. The deliverables that the contractor may be responsible for are not limited to the deliverables listed here.

1. A schedule of test participants indicating how they meet the criteria of the test protocol.

2. An activity report detailing the lab setup and how it meets the requirements of the test protocol and the audio and video recordings of the individual usability test sessions as specified in the test protocol.

PAGE 15 OF 98 SB1341-14-RP-0022

3. An activity report detailing the test materials and the test instrumentation and how it meets the requirements of the test protocol and the recorded measurements from these of the individual usability test sessions as specified in the test protocol.

4. An activity report detailing the tasks performed and a record of any data collected.

5. An activity report detailing the design how the final survey instrument was designed, how survey participants were identified, how surveys were distributed, the final survey instrument and the raw data collected.

6. An activity report detailing the tasks performed with respect to the interviews and contextual analysis and any and all raw data.

7. An activity report detailing the tasks performed to analyze the collected/raw data and the results of the analysis of the data.

8. An activity report recording the payment and compensation procedures for each participant.

9. An activity report detailing the progress of the work being performed and any pertinent issues that have arisen.

Standards for Acceptance of Deliverables: The standards for acceptance of deliverables will be specified in each individual task order. As a general rule the will review and reject or accept the deliverables within 10 business days of receipt. The contractor shall submit revised deliverables within 5 days of receipt of comments.

V. Reporting Requirements

Reporting requirements for each task will be specified in individual task orders.

VI. Period of Performance and Place of Performance

The period of performance of this IDIQ contract is for a base period of one year beginning on the date of award, and four (4) one year option periods.

All work under each task order shall be accomplished at the Contractor’s facility.

However, there may be some travel to the NIST Gaithersburg, MD campus, required in certain task orders.

VII. Government Furnished Property/Information

All data and deliverables generated under this contract remain the property of the Government.

NIST will provide the experimental protocol for all usability tests and any specific instrumentation (hardware/software) beyond the standard usability testing laboratory specified in specific task 2.

The COTR will be responsible for providing access to relevant NIST personnel.

VIII. Risk Assessment

PAGE 16 OF 98 SB1341-14-RP-0022

All work that shall be performed under this contract is designated as HSPD-12 Security Risk Level: Low Risk.

IX. Attachments

Example Test Protocol, Attachment (1)

X. Performance Requirements Summary

The following is a sample Performance Requirements Summary for a task order that would include all eight specific tasks detailed in this document. Each individual task order may have its own Performance Requirements Summary tailored to the requirements of the task order.

Task Desired Output

Required Service

Performance Standard

Monitoring Method

Acceptable Quality Level

Recruiting of Test Participants

A schedule of test participants

Recruit and screen test participants and manage scheduling of the usability tests.

Contractor personnel shall recruit and screen the required number of test participants according to the test protocol provided.

100% inspection by

COTR.

100%

Usability Testing Facilities

On site configured usability lab and audio/video recordings of test sessions

Contractor personnel shall configure the on-site usability lab according to the specifications of the test protocol to include screen capture and audio and video capture of each usability test session.

Contractor personnel shall record (audio/video and screen capture) of all usability test session— where a session corresponds to a participant according to the test protocol provided.

100% inspection by

COTR.

100%

Developing test materials and test instrumentation

Prepared test materials and instrumentation in place for the usability test sessions

Contractor personnel will develop test materials and any instrumentation in accordance

Contractor personnel shall provide the test materials and shall demonstrate that the instrumentation

100% inspection by

PAGE 17 OF 98 SB1341-14-RP-0022

with the specifications of the test protocol for use in the usability test sessions.

meets the specifications in the test protocol.

Carrying out or assisting in carrying out Test Protocols

Facilitated usability test sessions.

Contractor personnel will configure any data collection mechanisms or testing materials as specified in the test protocol during the experiment.

Contractor personnel shall document all data collected by the testing materials as specified by the test protocol

COTR.

100%

Analyzing data Completed data analysis

Contractor personnel shall perform data analysis in accordance with the test protocol

Contractor personnel shall provide the results of the data analysis

100% inspection by

COTR

100%

Proctoring of surveys

Distributed and proctored surveys

Contractor personnel shall distribute surveys in accordance with the test protocol.

Contractor personnel shall document how the surveys were distributed and provide the data collected by the survey instrument.

COTR.

100%

Facilitating interviews and contextual analysis

Facilitated and completed contextual analysis

Contractor personnel shall perform interviews and contextual analysis in accordance with the test protocol

– including taking notes and transcribing audio recordings.

Contractor personnel shall provide the interview and contextual analysis approach and data.

100% inspection by

COTR.

100%

Compensating test participants

Compensated test participants who have completed usability test sessions

Contractor personnel shall compensate test participants in accordance with the test protocol.

Contractor personnel shall provide a spreadsheet of the participant’s testing schedule and their compensation.

100% inspection by

PAGE 18 OF 98 SB1341-14-RP-0022

Project Management

Successful management of all specific tasks and assurance of timely provision of all acceptable deliverables.

Manage the quality and timeliness of all tasks/deliverables to ensure compliance with the Statement of Work and Test Protocol

Contractor personnel shall provide an activity report detailing the progress of work being performed and any pertinent issues.

100% inspection by

XI. Travel

Travel will be reimbursed under each individual task order in accordance with Federal Travel Regulation. The Contractor shall not exceed funding for travel as specified in each individual task order.

XII. Contractor-Government Interaction

All correspondence and oversight shall be provided by the Government through the Contractor’s designated individual. The Government will not provide direct or indirect supervision to any Contractor personnel.

XIII. Transition Plan

The Contractor shall submit a draft transition plan with their proposal. The draft transition plan should address how the contractor will transition to a subsequent contractor. The contractor shall explain how the data will be transferred to the Government and subsequent contractor.

Transition Report The Contactor shall submit to the Government three months prior to the contract expiration date a transition report. The transition report shall include the following information:

1. A cover page containing the contract number and title; the period of performance being reported; the Contractor's name, address, telephone, fax, and e-mail; the author(s); and date of submission;

2. A comprehensive list of all deliverables listed under the Reporting Requirements and Deliverables section of the contract, including data, web portals, databases, software applications and algorithms, SOPs, technologies, and any other resource generated under this contract;

3. A plan for the transportation of these resources and deliverables to a subsequent contractor or the Government, including timelines.

PAGE 19 OF 98 SB1341-14-RP-0022

INFORMATION FOR REVIEW OF RESEARCH

POTENTIALLY INVOLVING HUMAN SUBJECTS

Date of Submission:

Through:

Jane Doe: ________________________ Date: _________ [signature required] Group Manager, Visualization and Usability Group Information Access Division

John Doe: ___________________________ Date: _________ [signature required] Division Chief, Information Access Division Information Technology Laboratory

From: List all Principal Investigators and other investigators/researchers: [for each, specify name, degree, whether PI or other (post-doc, guest researcher, NIST employee or contractor, etc.), NIST address, telephone number, and e-mail address]

Jane Doe– NIST Employee, Principal Investigator Building 225, Room A225 Phone: (301) 975-XXX xxx@nist.gov

FAX: (301) 975-XXXX

1. Title of Project: Usability Testing: Fingerprint Directional Symbols

2. Site: Usability Laboratory at NIST

3. Estimated Duration of Study (including Experimental Schedule): 1 year, starting August 2012

4. Number of Participants: approximately 120

5. Types of Participants: (include selection criteria for subjects and method of recruitment of subjects; vulnerable subjects include people with mental or cognitive impairment, handicapped, economically disadvantaged, and other) NOTE: Research involving protected classes of subjects under 45 C.F.R. Part 46 Subpart B, C, or D, i.e., pregnant women, fetuses and human in vitro fertilization, prisoners, and children, may not be reviewed by the NIST IRB. Such research must be reviewed by an outside IRB that possesses current assurances, appropriate for the research in question, from OHRP.

Subjects will be recruited by Company A from the general population. Subjects will be between 19 and 79 years old, diversely distributed across gender, ethnicity, education and occupation. No protected classes of subjects will be recruited. NIST research team will

PAGE 20 OF 98 SB1341-14-RP-0022

administer the study and collect the data. Company A will schedule participants and make payment to participants.

6. Purpose of Project:

To study the feasibility of hand directional guidance in a language-independent format, i.e. symbolic, for unassisted fingerprint self-capture tasks.

7. Source of Funding (attach copy of any applicable MOU, IAA, contract, agreement, etc.):

8. Background and Objectives:

Currently biometric systems require operators to gather user’s biometrics. However, agencies would like to transition to unassisted biometric capture platforms. In order for these systems to be successful, users must be able to understand how to interact with the system and leave prints that have the appropriate quality within a reasonable amount of time. The guidance must be culturally independent and language independent. The NIST usability team has designed symbols to convey the directions for finger and hand placement as well as symbols to provide feedback to users. This protocol will examine the feasibility of hand directional guidance in a language-independent format, i.e.

symbolic, for unassisted fingerprinting tasks. Multiple variants of the directional symbols will be tested in an operational setting.

9. Experimental Protocol and Analysis Methods: (References for prior research in field may be added)

A graphical depiction of the tasks to be performed by NIST and Company A is attached.

There are eight hand directional concepts to be evaluated that are applicable to right slap (four fingers of the right hand), right thumb, left slap (four fingers of the left hand), and left thumb, as in Table 1. For each fingerprint, a concept has three symbol alternatives.

All subjects will perform a task on the fingerprint scanner by following the directional symbols displayed on a computer screen to guide them on moving their hands/fingers.

Half of the subjects will only perform the task with their right hands (right slap and right thumb), and the other half will only perform the task with their left hands (left slap and left thumb). So, each subject will go through 48 (=(8 +8) x3 alternatives)) directional symbols.

Table 1. Concepts for Fingerprint Directional Symbols Right slap Right thumb Left slap Left thumb Move up • • • • Move down • • • • Move right • • • • Move left • • • •

PAGE 21 OF 98 SB1341-14-RP-0022

Rotate clockwise • • • • Rotate counter-clockwise • • • • Press more • • • • Press less • • • •

There are four independent variables that will be measured:

• Response time – the time it takes from when a directional symbol is displayed until the subject moves his/her hand

• Correctness – whether the subject correctly follows the directional symbol to the desired position/placement; the subject’s hand movement will be recorded by the computer system as it’s shown on the computer monitor

• Interpretation – whether the subject correctly interpret the meanings of the directional symbols

• Preference – preferred symbol among the three alternatives for each concept

The fingerprint scanner will be placed on a table set to 39” high at a 20º angle, to simulate the current settings at various airports and based on the NIST height-angle study. An LCD monitor will be positioned on the table to display the directional symbols. A video camera will be positioned so that it only records the subjects’ hands.

Attach copy of any interview scripts, questionnaires and subject recruiting material (e.g., e-mail text, advertisements, posters, etc.). Consider whether any Paperwork Reduction Act (PRA) review/approval may be necessary (http://www-i.nist.gov/admin/mo/pra.htm).

• OMB approval under OMB Control Number 0693-0043 dated 5/17/2012 is attached.

• Recruiter Script is attached

Attach copy of all documentation of any required NIST safety/hazard review.

10. Contractor or Funding Recipient: If contractor or funding recipient involved, describe contractor/funding recipient proposed activity and attach copy of contract/funding document(s). Consult with the Office of the Chief Counsel for NIST, in advance, concerning required provisions/clauses for solicitation and/or contract.

11. Use of Existing Data/Material: If protocol involves use of pre-existing data/material, please describe it; identify/name the source/vendor (e.g., commercial vendor, subcontractor, university, national lab, etc.) for the data/material; and, if presently known, whether the data/material includes any personal identifiers that can be linked, directly or indirectly, to any individual.

PAGE 22 OF 98 SB1341-14-RP-0022

Not Applicable.

Attach copy of any applicable Material Transfer Agreement, MOU, contract, other agreement, etc., including any “click-through” terms/conditions for electronic acquisitions.

12. Summary of Human Subject Protocol:

Subjects will be recruited by Company A , from their database. Company A will schedule an appointment for the subject at NIST’s usability lab. The NIST research team (NIST PI and Company A PI) will administer the study and collect the data. Company A will schedule participants and make cash payment to participants before they leave NIST.

Subjects will participate in the study individually. The NIST research team member will act as test administrator and will greet each subject and inform them about the research.

The subject can ask any questions about the research at this time. Each subject will read/sign the consent form and receive a copy of the consent form. Each subject will be given a paper demographic questionnaire to fill out, that does not include any personal identifiers. Each subject will be assigned a subject number on the demographic questionnaire by the test administrator.

The test administrator will ask subjects to stand in front of the fingerprint scanning workstation, comprised of the finger print scanner and computer system with monitor, and perform a practice task of neutral hand positioning without any directional symbols, i.e. presenting right slap (all four fingers of the right hand) and right thumb, or presenting left slap (all four fingers of the left hand) and left thumb, based on the group assignment.

Subjects’ handedness will be collected from the demographic questionnaire and will be evenly assigned to the right-hand-only and left-hand-only groups. Half of the subjects will be assigned to the right-hand-only group, and the other half of the subjects will be assigned to the left-hand-only group. The assignment will be random except for handedness.

When the subject indicates that he/she is ready, the test administrator will start the test.

The computer system displays one directional symbol at a time, waits for the subject to respond on the finger print scanner, then displays the next symbol, until all 24 symbols for the slap and 24 symbols for the thumb are displayed.

After going through all 48 symbols once, the subject will be asked to fill out a post-task questionnaire with two parts. The first part asks the subject to interpret the meaning of each symbol from the task and the second part asks the subject to select the preferred symbol from the three alternatives for each concept. The test administrator will then measure the subject’s hand dimensions using a caliper and recording hand width and individual finger lengths.

PAGE 23 OF 98 SB1341-14-RP-0022

The task only simulates the fingerprint capturing process and no fingerprints will be actually captured. The subjects’ hands will be video recorded during the study. All of the data will be recorded without any personal identifiers. Each subject will be compensated by a staff person from Company A with $75 in cash for his/her time. If the subject withdraws before signing the consent form, or after signing the consent form but before completing the task, they will still be compensated for his/her time.

13. Potential Risks and Benefits:

There are no risks involved in this study. Potential benefits are improved fingerprint scanners and associated display systems.

14. Protection of Subjects against Potential Risks and Protection of the Subjects' Privacy:

Each subject’s data will be stored without personal identifiers. All data will be identified numerically only. In order to link together associated data, all of the subjects’ associated data will be assigned the same numerical classification. No key will be kept that links the numbers to the subjects’ individual identities.

15. Informed Consent Form: [if applicable, attach copy pursuant to Appendix C of NIST Administrative Manual Subchapter 14.01 (http://www-i.nist.gov/admin/mo/adman/1401appc.htm#Appendix%20C)]:

• Informed Consent Form is attached

16. CERTIFICATION:

“I hereby certify that the information furnished concerning the procedures to be taken for the protection of the human subjects is correct. I will seek and obtain prior Institutional Review Board (IRB) approval for any substantive modification of this protocol and will report any unexpected or otherwise significant adverse effects encountered in the course of this study."

Signature of Principal Investigator(s) and date

Signature of Principal Investigator(s) and date

PAGE 24 OF 98 SB1341-14-RP-0022

SECTION E

INSPECTION AND ACCEPTANCE

E. 1 52.246-4 INSPECTION OF SERVICES--FIXED-PRICE (AUG 1996)

(a) "Definitions." "Services," as used in this clause, includes services performed, workmanship, and material furnished or utilized in the performance of services.

(b) The Contractor shall provide and maintain an inspection system acceptable to the Government covering the services under this contract. Complete records of all inspection work performed by the Contractor shall be maintained and made available to the Government during contract performance and for as long afterwards as the contract requires.

(c) The Government has the right to inspect and test all services called for by the contract, to the extent practicable at all times and places during the term of the contract. The Government shall perform inspections and tests in a manner that will not unduly delay the work.

(d) If the Government performs inspections or tests on the premises of the Contractor or a subcontractor, the Contractor shall furnish, and shall require subcontractors to furnish, at no increase in contract price, all reasonable facilities and assistance for the safe and convenient performance of these duties.

(e) If any of the services do not conform with contract requirements, the Government may require the Contractor to perform the services again in conformity with contract requirements, at no increase in contract amount. When the defects in services cannot be corrected by reperformance, the Government may--

(1) Require the Contractor to take necessary action to ensure that future performance conforms to contract requirements; and

(2) Reduce the contract price to reflect the reduced value of the services performed.

(f) If the Contractor fails to promptly perform the services again or to take the necessary action to ensure future performance in conformity with contract requirements, the Government may--

(1) By contract or otherwise, perform the services and charge to the Contractor any cost incurred by the Government that is directly related to the performance of such service; or

(2) Terminate the contract for default.

(End of Clause)

PAGE 25 OF 98 SB1341-14-RP-0022

SECTION F

DELIVERIES OR PERFORMANCE

F. 1 52.242-15 STOP-WORK ORDER (AUG 1989)

(Reference 52.242-15)

F. 2 52.242-17 GOVERNMENT DELAY OF WORK (APR 1984)

(Reference 52.242-17)

F. 3 1352.270-70 PERIOD OF PERFORMANCE (APR 2010)

(a) The base period of performance of this contract is from TBD through TBD . If an option is exercised, the period of performance shall be extended through the end of that option period.

(b) The option periods that may be exercised are as follows:

Period Start Date End Date Option I TBD TBD Option II TBD TBD Option III TBD TBD Option IV TBD TBD

(c)The notice requirements for unilateral exercise of option periods are set out in FAR 52.217-9.

PAGE 26 OF 98 SB1341-14-RP-0022

SECTION G

CONTRACT ADMINISTRATION DATA

G. 1 1352.201-70 CONTRACTING OFFICER?s AUTHORITY (APR 2010)

The Contracting Officer is the only person authorized to make or approve any changes in any of the requirements of this contract, and, notwithstanding any provisions contained elsewhere in this contract, the said authority remains solely in the Contracting Officer. In the event the contractor makes any changes at the direction of any person other than the Contracting Officer, the change will be considered to have been made without authority and no adjustment will be made in the contract terms and conditions, including price.

G. 2 1352.201-72 CONTRACTING OFFICER?s REPRESENTATIVE (C0R) (APR 2010)

(a) Fred Byers is hereby designated as the Contracting Officer?s Representative (COR). The COR may be changed at any time by the Government without prior notice to the contractor by a unilateral modification to the contract. The COR is located at:

TBD

TBD

TBD

Phone Number: TBD Email: TBD

(b) The responsibilities and limitations of the COR are as follows:

(1) The COR is responsible for the technical aspects of the contract and serves as technical liaison with the contractor. The COR is also responsible for the final inspection and acceptance of all deliverables and such other responsibilities as may be specified in the contract.

(2) The COR is not authorized to make any commitments or otherwise obligate the Government or authorize any changes which affect the contract price, terms or conditions. Any contractor request for changes shall be referred to the Contracting Officer directly or through the COR. No such changes shall be made without the express written prior authorization of the Contracting Officer.

The Contracting Officer may designate assistant or alternate COR(s) to act for the COR by naming such assistant/alternate(s) in writing and transmitting a copy of such designation to the contractor.

G. 3 1352.216-76 PLACEMENT OF ORDERS (APR 2010)

(a) The contractor shall provide goods and/or services under this contract only as directed in orders issued by authorized individuals. In accordance with FAR 16.505, each order will include:

(1) Date of order;

(2) Contract number and order number;

(3) Item number and description, quantity, and unit price or estimated cost or fee;

(4) Delivery or performance date;

(5) Place of delivery or performance (including consignee);

(6) Packaging, packing, and shipping instructions, if any;

(7) Accounting and appropriation data;

(8) Method of payment and payment office, if not specified in the contract;

(9) Any other pertinent information.

(b) In accordance with FAR 52.216-18, Ordering, the following individuals (or activities) are authorized to place orders against this contract:

NIST Contracting Officer N/A

(c) If multiple awards have been made, the contact information for the DOC task and delivery order ombudsman is N/A

PAGE 27 OF 98 SB1341-14-RP-0022

G. 4 1352.245-70 GOVERNMENT…