v2 Attachment 1 PWS Data Training 20Q5000.pdf

DOS/FSI/EX/ACQ

United States Department of State Foreign Service Institute

School of Applied Information Technology (SAIT) Performance Work Statement (PWS)

DOS/FSI/SAIT/BA requires a vendor to design and deliver interactive workshops to expand data literacy for Department employees who produce and consume data.

Background The U.S. Department of State’s Foreign Service Institute (FSI) is the bureau of record responsible to provide training opportunities in support of the Department’s mission. The School of Applied Information Technology (SAIT) is the school within FSI responsible for conducting IT training for the entire Department. In support of Department of State goals to increase employee data literacy, SAIT provides training on a variety of data literacy topics.

SAIT has offered data literacy courses for the past three years, as a mix of in-house and vendor-led offerings. SAIT’s in-house instructional staff teach a course called “Data Analysis and Visualization” to employees. Students use Microsoft Excel and Microsoft Power BI for class exercises and are introduced to other software tools and processes during discussion and presentations delivered by members of the State Department’s Data Community.

More recently SAIT has contracted out teaching for training on 4 additional courses: 1) a course on programming in R Studio, 2) a course on data literacy topics for managers, 3) a course on data literacy topics for executives, and 4) an advanced course on data modeling using Excel and Power BI. These courses complement SAIT’s in-house offerings.

Purpose FSI is seeking subject matter experts to design and deliver highly interactive courses and seminars to expand data literacy and data analytics training for Department employees. The intent is to ensure SAIT is offering courses that teach to the most current and relevant data literacy principles and tools by tapping outside expertise in the data field that can provide additional data literacy, data analysis, and data visualization workshops and/or seminars to a broad spectrum of Department employees, in a variety of delivery modalities and on a range of topics.

Scope The Department of State’s mission covers a vast spectrum of functional work and foreign policy activities (www.state.gov). To effectively accomplish these activities, the Department has positioned the objective of enabling data-informed decisions at all levels as a key organizational goal. SAIT’s mandate is to support this goal by providing training to Department employees which teaches them data principles, processes, and tools that form the foundation of a data-driven culture.

We are therefore looking to offer a repeatable series of courses delivered to Department employees. We are looking for vendors who can provide training that meets the learning objectives in our existing vendor-led courses, while also offering 2 additional courses in new material. We invite vendors to propose response for the workshops described below. We expect to offer each workshop between 3 and 12 times a year, though the actual number of offerings may vary, and may increase, based on need.

We are interested in courses that can be delivered to a minimum of 12 and no more than 25 employees, as well as programmatic data events that would be offered to larger audiences.

Depending on the Department’s need, courses will be delivered in:

file://fsiexacqfp01/ACQ_Staff_Share/Solicitations/FY18/Online%20Labs%20RFI%20for%20SAIT/Draft%20PWS/www.state.gov

1) Face-to-face sessions in physical environments at State Department locations. Unless otherwise specified classes will be held at the Foreign Service Institute (FSI) in Arlington, VA and times listed are for Eastern Standard Time. However, some opportunity may exist to for classes to be delivered at other locations in the Washington, D.C. region, or at Department locations overseas and other time zones.

Sessions will be either one or multiple full days or half days in duration.

Vendors are asked to provide digital copies of any class materials to SAIT at least 5 days prior to the start of class so SAIT may include them in registration communications with students. In the event vendor needs to provide hard copy materials, vendor will produce the materials and provide them in advance.

For in-person classes at FSI, vendor should arrive to the training site a minimum of 45 minutes before the start of the class. If the instructor is new to the facility, they may request an on-site visit to familiarize themselves with the classroom setup and equipment. If the instructor is local to the Washington D.C., area they should request a site visit at least 3 days before the class start date, in case there are any issues that need technical support to resolve. If the instructor is not local, they may request a site visit the day before the class start and may also request a virtual site visit and consultation via web conferencing with the Program Manager or Course Coordinator. The on-site visit is considered part of course preparation and is not billable separately.

2) Synchronous live-virtual sessions through web conferencing systems. Unless otherwise specified, class times are in Eastern Standard Time, though there will be opportunity to deliver classes to students in other time zones. Vendors should specify any additional factors that should be considered in offering classes in other time zones.

Sessions will be either one or multiple full days or half days in duration.

For virtual classes, vendor is asked to login to the virtual meeting space a minimum of 30 minutes before the start of class.

Vendors are asked to provide a course syllabus for the class, an instructor bio, and digital copies of class materials to SAIT at least 5 days prior to the start of class so SAIT may include them in registration communications with students. For in-person classes, vendors are asked to bring hard copies of materials and handouts with them to the training site.

With virtual delivery there will be no opportunity to provide hard copy materials. For online class components, vendors are asked to provide the platform(s), with the understanding that they must be approved for use by SAIT before being used with Department employees.

Courses are listed as full day but may be reconfigured as half-day sessions, with two half days being the equivalent of one full day. Sessions will be conducted in Eastern Standard Time unless specified, but there may be occasional need for sessions that can accommodate a student population from a different time zone, such as time zones in the East Asia Pacific region.

Where listed as a full day, classes at FSI run from 9 a.m. to 4 p.m. with a 45-minute lunch break and two 15-minute breaks each day. To extend the training to a full 8-hour day, we require that awarded vendor(s) make available a 1 hour tutorial or advisory session following the conclusion of each day of the formal workshop during the 4-5 p.m. timeslot for any employee who wishes to stay and ask for additional help.

Where listed as half days, classes at FSI run from 9 a.m. to noon with a 15-minute break at approximately the 90-minute mark. To extend the training to a half day, we require that awarded vendor(s) make available a 1 hour tutorial or advisory session following the conclusion of each day of the formal workshop during the 12-1 p.m. timeslot for any employee who wishes to stay and ask for additional help.

Programmatic events may be either in person or online depending on the content and objective.

Existing Vendor-Led Courses

Course 1: Programming with R

Workshop Duration: 5 Full Days, or 10 Half-Days

• Perform the foundations and effective use of R programming,

• Install and configure R for statistical programming environment,

• Load, clean, and visualize data in R,

• Demonstrate how to apply algorithms to build predictive models using R,

• Create static and interactive visualizations using R, and

• Demonstrate R’s functionality through connectivity to R packages, and other platforms, and software.

Department employees who produce data analysis and data visualization are attracted to “hands-on” and practical “how-to” workshops. Department employees at the analyst level have experience with analyzing subject matter policies, field reporting and other contextual or text-based analyses but need to learn how to incorporate data analysis and best practices into their data visualizations. Building on the existing “Data Analysis & Visualization” class SAIT already offers, these Department employees are eager to learn new tools and take their Excel proficiency to the highest level while others are ready to learn and effectively use the R programming language. They want to become more familiar with data analysis and visualization concepts, best practices, and strategies to create effective visualizations and to tell stories with data.

Course 2: Data Literacy for Managers

Workshop Duration: 2 Full Days, or 4 Half-Days

Non-tool training designed for managerial audiences. Employees who attend this class will have the following performance outcomes:

• Describe the benefits of data-driven strategies within organizations,

• Apply principles of valid data analytics, data visualization, and data science,

• Develop a framework for questions to which data analytics can be applied,

• Identify the hallmarks of effective data storytelling,

• Identify foreign policy-related data sources,

• Discuss best practices from other data-driven organizations, and

• Identify issues surrounding data ethics and cognitive bias to avoid data pitfalls and improper use of data and data analytics.

Managerial-level staff need to understand how to navigate and query data presentations, as well as how to interpret data analysis to better inform their decision-making processes and to create compelling data stories of their own to justify the need for resources and/or to negotiate a particular policy. Department employees at the level are consumers of data analysis and are attracted to training events that would provide inspiring examples of data analysis and data visualization as effective and essential drivers influencing strategic resource planning and policy making. Principles of data analysis, trend analysis, high-level overviews of predictive modeling, and knowing how to recognize and avoid data pitfalls such as data bias are all topics that would help leaders build a more data-driven organization.

Course 3: Data Literacy for Executives

Workshop Duration: 1 Full Day, or 2 Half-Days

Non-tool training designed for executive audiences. Employees who attend this class will have the following performance outcomes:

• Describe the benefits of data-driven strategies within organizations,

• Develop a framework for questions to which data analytics can be applied,

• Identify the hallmarks of effective data storytelling,

• Discuss best practices from other data-driven organizations, and

• Identify and discuss issues surrounding data ethics and cognitive bias to avoid data pitfalls and improper use of data and data analytics.

Executive-level staff need to understand how to interpret and leverage data presentations, as well as how to interpret data analysis to better inform their decision-making processes to negotiate a particular policy. Department employees at the executive level are consumers of data analysis and are attracted to training events that would provide inspiring examples of data analysis and data visualization as effective and essential drivers influencing strategic resource planning and policy making. Principles of data analysis, trend analysis, high-level overviews of predictive modeling, data ethics, and knowing how to recognize and avoid data pitfalls such as data bias are all topics that would help leaders build a more data-driven organization.

Course 4: Advanced Analysis and Modeling with Excel & Power BI

Workshop Duration: 3 Full Days, or 6 Half-Days

• Use advanced-level features of Microsoft Excel,

• Execute statistical and predictive models in Excel,

• Create static and interactive visualizations using Excel, and

• Demonstrate Excel’s functionality through connectivity to Microsoft’s Power BI and other platforms and software.

Tool training designed for employees already familiar with Excel and those who are practitioners of data analysis and looking to take their skills to a more advanced level. These employees would already be proficient with Excel and regularly perform data analysis in their work.

New Vendor-Led Courses

The following two courses are new courses that are being solicited based on the Department’s evolving needs for data literacy and data analytics training.

Course 5: Fundamentals of Data Literacy

Duration: 2 Full Days, or 4 Half-Days

This course is designed for Department employees who do not work in data analytics areas, to educate them on the fundamentals of data sciences for non-practitioners. Topics covered will include fundamental data science concepts and associated terminology; why data science is important; what it means to work in a data-driven culture; skills necessary to thinking critically about data; common issues in data collection and analysis such as bias, data gaps and imprecision; strategies for interpreting data visualizations produced by others; and basic steps those who are not data scientists can take to incorporate data analysis into their own work. This is a non-tools course that focuses on concepts and principles. Upon course completion, employees who take this course will be able to do the following:

• Describe why data, and a data-driven culture, is important in a modern organization,

• Define fundamental data science concepts and terminology,

• Use critical thinking strategies to accurately formulate and frame business problems,

• Identify and know how to avoid common mistakes in gathering data and performing data analysis, and

• Evaluate whether data visualizations provided by others are well-designed and accurately portray the data.

Many Department employees do not see how data science applies to them because they work with data infrequently, they do not work in analyst positions, they have preconceived notions of what "real data" looks like, or they are intimidated by the terminology. In an effort to build a data-driven culture at the Department of State it is important that every employee realize that the business or policy data they work with every day is vital to a data-driven culture and understand how they can connect with it.

Course 6: Data Visual Design and Storytelling Duration: 2 Full Days, or 4 Half-Days

This course is designed to teach Department employees the principles behind creating effective visualizations, interpreting visualizations provided by others, and crafting narratives that can serve as the voice for the data and put the data into context. Topics covered will include the basics of interpreting different types of charts; how to spot misleading charts and graphs provided by others; principles of design to use when creating one's own visualizations, including choosing chart types that match the data type, strategies for reducing cognitive load, avoiding design choices that can obscure data; using design elements to create emotion and motivation; and principles for crafting a narrative that puts the data in context, considers the audience for the message, and provides actionable insights. This course is focused on concepts, and should not be tied to a data analytics tool, though hands-on activities should where possible have participants use PowerPoint to analyze and modify existing charts. Upon course completion, employees will be able to do the following:

• Recognize misleading or inaccurate charts and graphs provided by others,

• Understand the design principles involved in creating effective and accurate visualizations, regardless of the data tool, and

• Create a narrative that accurately supports the data, provides context, and reveals actionable insights.

Employees at the Department of State work with copious amounts of visualizations. Some charts and graphs are provided by others, some are created internally. However, many employees are not taught the foundational design principles behind good visualizations. This leaves them vulnerable to using visualizations that at best are wrong and at worst are misleading, or creating visualizations that either do not accurately reflect the data or are hard to read by others. Even when visualizations are well-designed, there is not a culture of using the data and the visualizations as part of a bigger story, to drive decision-making based on context rather than just numbers. Employees who understand the principle of good visualizations and good data storytelling can more effectively support a data-driven culture and driven decision-making.

Department of State Programmatic Data Events

Programmatic Data Events are designed to advance and support the State Department Data Program. Such events may include, but are not limited to, topic-specific webinars, hackathons, talks, workshops, and other events which advance the Department’s understanding of and competency with data literacy.

Employees who attend such events will have of the following performance outcomes:

• Increased awareness of industry trends, best practices, and emerging topics, and

• Increased awareness of key concepts in data literacy and data analytics and its importance to State Department strategic objectives and operations.

Department of State Programmatic Events are designed to reach a greater audience across the State Department enterprise and advance the data literacy program and data literacy throughout the Department. They will be developed and offered on an as-needed basis to build upon and deepen State Department data competency. They may be added virtually or in-person in the Washington, D.C. area depending on the needs of the Department.

In the event of multiple awardees, fair opportunity shall be provided to deliver ad hoc Programmatic Data Events. FSI will share with the awardee(s) a topic, specific objectives (if applicable) and event times and dates, and ask each awardee to provide a quote to include a proposed workshop itinerary, proposed instructor, and quoted price to support the topic selected by SAIT. Pricing shall not exceed the established daily rate for the applicable ordering period. DOS shall review all quotes received by awardee(s) for the specific proposed data event and select the quote/awardee that aligns best with data event objectives considering the proposed itinerary, instructor, and quoted price.

Examples of contemplated data events include but are not limited to:

Event Type 1: Webinar on Advanced Topics Workshop Duration: 2 hours

As the Department moves to a data-driven culture, there is a need for Employees at all levels to be familiar with emerging topics in areas of data analytics that may support either the business or policy goals of the Department. For most employees, the knowledge they would need or be interested in is at a high level, which can be covered in a shorter, non-interactive talks for large audiences. To meet this need, Data Talks will be 2-hour lecture-based talks delivered to audiences of up to 100 attendees, via a virtual platform. We expect to offer approximately one webinar per quarter, but actual frequency and rotation of topics may vary.

Topics should be on advanced topics but structured in content and comprehension level to the layperson who does not yet have a working knowledge of the topic. Topics may include:

• Fundamental Concepts in Artificial Intelligence,

• Fundamental Concepts in Machine Learning,

• Fundamental Concepts in Natural Language Processing, and

• Fundamental Concepts in Geography-Based Data Analytics.

Upon completion of the webinar, employees will be able to do the following:

• Describe fundamental concepts or issues in the topic area.

Event Type 2: Data Hackathon

Hackathons are a proven way to generate lots of ideas to problem-solve. They have become a staple in tech fields and beyond both to problem-solve and to raise awareness of pertinent issues. SAIT is interested in hosting a virtual hackathon on a yet-to-be determined topic using a publicly-available Department dataset. The model would draw on how the Department of State has conducted previous hackathons. Participants may be internal or external to the Department depending on the design of the hackathon. Upon completion of the event, participants will have a greater awareness of the role data plays at the Department of State.

Performance Requirements

1. Curriculum Design & Delivery: The contractor shall be tasked with designing, developing, and delivering the curriculum of the workshops. A final curriculum should be approved by FSI no later than 10 business days before the first offering of a course, or in the event of major changes to the course, 10 business days before the first use of the content in a class.

2. Collaborative Meetings: The contractor will participate in collaborative meetings (in person or via telephone) with FSI staff to review and refine curriculum content and delivery plans. The content of meetings includes refining design plans, integrating workshop feedback, and guiding learning scenarios that align with the context of FSI.

These meetings will not be billed separately so the cost shall be included in the fully burdened firm fixed session rates.

3. Technical Specifications:

a. In classroom environments, students will have access to PCs with Windows 10 operating system and Microsoft Office 2016 (including Excel), and Microsoft Power BI as well as internet connection to view online sites, though they will not be allowed to download new software. In the case of R training, students will have access to computers that have R Studio installed and can connect to an alternate network if needed. The contracted instructor(s), when working on premises, will have access to a Department of State unclassified computer and/or be able to connect his/her laptop to the audio/visual equipment in the classroom. Both wi-fi and wired connections to the internet are available.

b. When delivered online, FSI cannot guarantee the technical specifications of the computers that students will have access to, but will request that students have access to Excel, Power BI local desktop application, and if applicable and permitted by the Department, the Power BI services online application. For R classes, FSI requests that the vendor provide access to RStudio Cloud or other comparable platform for students to use R Studio virtually. For any internet-based software component, including tools for teaching via web conferencing, vendors are asked to provide the platform(s), with the understanding that they must be approved for use by SAIT before being used with Department employees.

4. Course Materials:

a. If electing to provide hard copy material to students, vendor shall provide them in advance. Cost of materials is not separately billable from the session rate.

5. Evaluation: FSI evaluates from Level 1 through Level 3 in the Kirkpatrick New World

Model. Level 1 is reaction to the training; Level 2 is knowledge acquisition; and Level 3 is knowledge application. FSI provides Level 1 evaluation through a standard survey given to all students at the end of training and provides Level 3 evaluation through a survey to students 3 months after training. Level 1 results will be provided to the vendor after each class, and level 3 results will be provided to the vendor at the time that FSI collects that data. Vendors are expected to use the Level 1 and Level 3 results to update and modify curricula.

For Level 2, per a new FSI requirement, vendors will be expected to develop and deliver a Level 2 evaluation for each course. Evaluation content must tie back to what is taught in the course content and should be provided before the beginning of training after course content has been developed and finalized with the COR. Evaluation may be conducted by having students perform tasks to demonstrate knowledge, or by answering a final exam to demonstrate knowledge. FSI prefers a task-based approach, but will accept either methodology provided the evaluation is based on measuring knowledge acquired during the course. Evaluation must be done as part of class time.

If evaluating with an exam, vendor may use an online system such as a form or survey software that does automatic grading, but the software should not record full names of students or any other PII, and exam results should not be saved.

If the vendor assesses learning via task performance, the student’s grade as assessed or observed by the instructor is as follows:

o Student was able to complete tasks with minimal assistance: 100%, passing o Student was able to complete tasks with assistance: 70%, passing o Student was not able to complete tasks with assistance: 69% failing

If the vendor assesses learning via exam, the final exam must contain a minimum of 10 multiple choice questions, and a maximum of 20 multiple choice questions. Each question should have 3-5 answer choices, with only one correct answer. A grade of 70% or above is a passing grade.

Regardless of assessment method, the instructor must record the score on the class roster and return it to the course coordinator at the completion of class. This score is recorded separately from the complete/incomplete requirement, which only records attendance.

Instructor Qualifications Each instructor included in the technical proposal shall have the following professional experience and skills: experience in performing Data Literacy, Analysis and Visualization work. Demonstrated understanding of Foreign Affairs work. Experience teaching adult learners in this field. Experience teaching this field to the Federal workforce.

Operating Constraints While the students will have access to the internet, they will not be able to download software unapproved by the Department of State for use during delivery of the workshops. All software must be approved in advance of the workshops.

Place of Performance/Delivery For classroom-based courses, the contractor will deliver instruction to employees in a Department location in the Washington, D.C. area. Unless specified otherwise, classroom courses will be held in a classroom or lecture hall at the Department of State’s training campus: 4000 Arlington Blvd. Arlington, VA 22204.

Department of State George P. Shultz National Foreign Affairs Training Center Shultz Center FSI Washington, DC 20522-4201

Virtual courses will be held online, and the instructor may teach from any location that has strong internet access. Government does not provide internet access or wi-fi in the case of virtual training.

Government-Furnished Property When working on premises, a Department of State unclassified computer will be provided and/or the contractor will be able to connect his/her laptop to the audio/visual equipment in the classroom.

Fully Loaded Firm Fixed Price Daily Rate Daily rates include all required preparation plus delivery of one training day to include the 1 hour of on-site availability for advisory/tutorial services at the end of each training day.

The Contractor is entitled to bill for the number training days as applicable for duration of the delivered course, regardless of the number of participants (no less than 12 and no more than 25).

Session rates provided by the contractor shall be fully loaded and inclusive of all costs for which the contractor expects compensation, including travel to and from the place of performance in Arlington, VA. No other costs (including course development and meetings) shall be billed separately.

Security Clearance Requirements None.

Anticipated Period of Performance September 1, 2020 to August 31, 2025.