19FS1A20N5000 Data Training DRAFT PWS.pdf

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been developed by the Government for market research purposes only. Nothing in this document can be construed to bind the Government in any way. A formal solicitation for this requirement may follow.

DOS/FSI/EX/ACQ

United States Department of State Foreign Service Institute

School of Applied Information Technology (SAIT) Draft Performance Work Statement (PWS)

DOS/FSI/SAIT/BA FSI is seeking a subject matter expert contractor to design and deliver highly interactive workshops, sessions, and seminars that can complement, update, or expand the existing instruction SAIT offers for Department employees who need to collect, analyze, and visualize data as practitioners or analyze and use data to frame or formulate decisions.

DRAFT Performance Work Statement (PWS) Data Literacy, Analysis & Visualization Training

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

Background The U.S. Department of State’s Foreign Service Institute (FSI) is the bureau of record responsible to provide training opportunities in support of the Department’s mission. The School of Applied Information Technology (SAIT) is the school within FSI responsible for conducting IT training for the entire Department. In support of Department of State goals to increase employee data literacy, SAIT provides training on a variety of data literacy topics.

For the past three years, SAIT in-house instructional staff have taught a course called “Data Analysis and Visualization” to employees. Students use Microsoft Excel and Microsoft Power BI for class exercises and are introduced to other software tools and processes during discussion and presentations delivered by members of the State Department’s Data Community. Demand for this introductory-level class is high, and employees performing a wide spectrum of functional work and serving in roles from entry level to senior level have expressed interest in receiving additional data literacy training. To help fill this demand, more recently SAIT has contracted out teaching for training on R Studio, data literacy topics for managers and executives, and an advanced course on data modeling using Excel and Power BI. These courses complement SAIT’s in-house offerings.

Purpose FSI is seeking subject matter experts to design and deliver highly interactive workshops, sessions, and seminars that can complement, update, or expand the existing instruction SAIT offers for Department employees who need to collect, analyze, and visualize data as practitioners or analyze and use data to frame or formulate decisions. The intent is to ensure SAIT is offering courses that teach to the most current and most useful data literacy principles and tools by tapping outside expertise in the data field that can provide additional data literacy, data analysis, and data visualization workshops and/or seminars to a broad spectrum of Department employees, in a variety of delivery modalities and on a range of topics.

Scope The Department of State’s mission covers a vast spectrum of functional work and foreign policy activities (www.state.gov). To effectively accomplish these activities, the Department has positioned the objective of enabling data-informed decisions at all levels as a key organizational goal. SAIT’s mandate is to support this goal by providing training to Department employees which teaches them data principles, processes, and tools that form the foundation of a data-driven culture. We are therefore looking to offer a range of training delivered to two difference audiences: practitioners who produce data analysis and executives and managers who consume data analysis to inform decision and policy making. We are looking to explore what types of training exist for both, in a variety of formats and modalities.

The following descriptions are examples to help describe what we are looking for but should not limit the possible training scenarios.

Audience Descriptions: Producers of Data file://fsiexacqfp01/ACQ_Staff_Share/Solicitations/FY18/Online%20Labs%20RFI%20for%20SAIT/Draft%20PWS/www.state.gov

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

Department employees who produce data analysis and data visualization fall into a few different categories:

Producers of Data: Basic General Data Practitioners These employees will conduct data tasks periodically as a part of their job. Data tasks may not occur frequently, but when they occur, they will be part of fast-turnaround data calls and may span a variety of subjects. Practitioners will be expected to collect the data, likely from disparate sources and formats, organize it, clean it, and fashion it into visualizations that easily and accurately convey information. These employees should also have a basic understanding of the general concepts behind Machine Learning, Artificial Intelligence, and Natural Language Processing as a part of general data literacy skills.

Skills that might fall into this role include, but are not limited to:

• Collecting data

• Checking the accuracy of data

• Data cleaning

• Using basic data manipulation techniques such as Excel formulas and PivotTables

• Choosing the visualization that best complements the product of a data analysis

• Designing clear and effective charts/graphs in Excel and Power BI

• Choosing the type of visualization that best highlights the data and the message

• Ability to explain general data literacy concepts

These employees would most likely use Excel and Power BI to complete these tasks. They would benefit from “hands-on” and practical “how-to” workshops that combine basic data literacy principles with knowledge of how to use those principles on the job.

Department employees at this level need to learn how to incorporate data analysis and best practices in data visualization into their repertoire. These entry and mid-level Department employees need to learn the fundamentals of data analysis, how to overcome common challenges associated with identifying data sets, how to clean and organize data in order to determine trends and patterns, and how to mitigate data bias. With those fundamentals in place, they need to understand how to create effective, accurate, and visually clear charts and graphs.

Producers of Data: Experienced General Data Practitioners These employees would also be expected to gather data, clean it, and put into appealing charts and graphics that tell a story. Unlike basic data practitioners, data tasks may be a more frequent part of their job and they may need to bring in data from a wider variety of sources, work with larger data sets, perform advanced data cleaning, create complicated graphics or infographics, and develop interactive dashboards. These practitioners have already mastered the skills of a basic data practitioner.

Skills that might fall into this role include, but are not limited to:

• Setting up recurring/continuous data processes

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

• Integrating data from multiple sources

• Employing advanced data cleaning processes

• Performing advanced data manipulation techniques

• Designing advanced charts, reports and dashboards

• Telling data stories with advanced visualizations

• Identifying open-source data repositories, analysis, and visualizations relevant to the field of foreign policy

• Understanding how Machine Learning, Artificial Intelligence, and Natural Language

Processing could be applied to their data work

Tools that this type of practitioner might use include, but are not limited to, Excel, Power BI, and Tableau.

Department employees at this level have already mastered foundational data literacy principles. They would most benefit from training that focuses on advanced principles and practices around collecting data, cleaning data, working with large data sets and creating advanced visualizations. They may also benefit from tools courses that focus on advanced tools or methodologies, and on training that shows them how leverage Machine Learning, Artificial Intelligence, and Natural Language Processing in their data work.

Producers of Data: Specialized Practitioners These employees either work with data as a core part of their job, or work with specialized data sets. They may have to perform complex operations on large sets of data or perform complex tasks to acquire the data. They have already mastered the skills of basic and advanced data practitioners and are extremely well-versed in foundational data literacy concepts.

Skills that might fall into this role include, but are not limited to:

• Data mining and scraping

• Acquiring external data sets

• Manipulating very large data sets

• Working with specialized data sets such as geographic data or large sets of longitudinal data

• Identifying specialized open-source data repositories, analysis, and visualizations

Tools that this type of practitioner might use include, but are not limited to, Excel, Power BI, Tableau, Python, R Studio, and ArcGIS.

Department employees at this level are already knowledgeable about data principles and practices. They would benefit from training that focuses on specialized tools, and training that shows them how to leverage Machine Learning, Artificial Intelligence, and Natural Language Processing in their data work.

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

Audience Description: Consumers of Data Analysis

Consumers of Data: Managers & Executives Consumers of data analysis are Department employees that use data to frame or formulate decisions. They are sometimes the producers of the data. Often these employees are managers and executives who either use the data products to formulate a decision or are presenting the data products to other managers and executives. They must be able to accurately digest the data in presentations, detect flaws and data bias in visualizations, interpret data analysis to better inform their decision making processes, and construct compelling narratives with the data in order to justify the need for resources and/or negotiate a particular policy.

Skills that might fall into this role include, but are not limited to:

• Reading and evaluating charts and graphs

• Awareness of how the choice of visualization type can affect the perception of the data

• Recognizing data bias

• Familiarity with heuristics and methodologies common in evaluating data, such as problem framing and cost/benefit analysis

• Constructing narratives with data based on visualizations provided by producers

• Understanding how Machine Learning, Artificial Intelligence, and Natural Language

Processing can be used to create a data-driven culture

Existing/Legacy Training for Producers and Consumers of Data Analysis

Existing Training for Producers of Data Analysis Department employees at the analyst level have experience with analyzing subject matter policies, field reporting and other contextual or text-based analyses but need to learn how to incorporate data analysis and best practices into their data visualizations. They are familiar with Excel and have taken the existing “Data Analysis & Visualization” class SAIT offers in-house. To meet the needs of this employee population, we currently offer the following vendor-led trainings:

Training 1: Business Data Analysis: Advanced Analysis and Modeling with Excel & Power

BI

Training Duration: 3 days

Tool training designed for analyst audiences. Employees who attend this class will have the following performance outcomes:

• Use advanced-level features of Microsoft Excel.

• Execute statistical and predictive models in Excel.

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

• Visualize complex data sets using Excel & Power BI.

• Demonstrate Excel’s functionality through connectivity to Microsoft’s Power BI and other platforms and software.

Training 2: Business Data Analysis: Programming with R

Training Duration: 5 days

Tool training designed for analyst audiences. Employees who attend this class will have the following performance outcomes:

• Perform the foundations and effective use of R programming

• Install and configure R for statistical programming environment

• Load, clean, and visualize data in R

• Demonstrate how to apply algorithms to build predictive models using R

• Create static and interactive visualizations using R

• Demonstrate R’s functionality through connectivity to R packages, and other platforms, and software

Existing Training for Consumers of Data Analysis Employees at the managerial and executive levels need to understand how to navigate data presentations, how to interpret data analysis to better inform their decision-making processes, and how to create compelling data stories of their own to justify the need for resources and/or to negotiate a particular policy. These employees are focused on using data for decision-making and instilling the principles of a data-driven organization in their business units. To meet the needs of this employee population, we currently offer the following vendor-led trainings:

Training 3: Data Literacy for Managers

Training Duration: 2 days

Non-tool training designed for managerial audiences. Employees who attend this class will have the following performance outcomes:

• Describe the benefits of data-driven strategies within organizations

• Apply principles of valid data analytics, data visualization, and data science

• Develop a framework for questions to which data analytics can be applied

• Identify the hallmarks of effective data storytelling

• Identify foreign policy-related data sources

• Discuss best practices from other data-driven organizations

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

• Identify issues surrounding data ethics and cognitive bias to avoid data pitfalls and improper use of data and data analytics

Training 4: Data Literacy for Executives

Training Duration: 1 days

Non-tool training designed for executive audiences. Employees who attend this class will have the following performance outcomes:

• Describe the benefits of data-driven strategies within organizations

• Develop a framework for questions to which data analytics can be applied

• Identify the hallmarks of effective data storytelling

• Discuss best practices from other data-driven organizations

• Identify and discuss issues surrounding data ethics and cognitive bias to avoid data pitfalls and improper use of data and data analytics

New Training Offerings for Producers and Consumers of Data

New Courses in Data Literacy, Data Analytics and Data Visualization

Duration: In a classroom setting, not to exceed 3 days of seat time. The duration of virtual alternatives may vary, provided that material equivalent to the classroom version is covered.

SAIT envisions new training offerings to complement existing courses. and expects that approximately 3-5 new courses will be chosen to complete the current data literacy and analytics curriculum, depending on the topics chosen and duration of training offered.

Vendors should suggest potential training topics, formats, and durations for consideration, with the understanding that each approved course will be offered as a classroom course with the option for online delivery as well. SAIT believes that courses in the topics listed below may be most beneficial to a Department audience, but this is not an exhaustive list and is meant to provide examples to be refined based on vendor capabilities and responses. Vendors may suggest additional/alternative topics based on their knowledge and experience with industry trends.

• Designing effective and informative visualizations

• Accurately reading and evaluating visualizations produced by others

• Data storytelling

• Cleaning and organizing data

• Understanding the basic principles of Machine Learning, Artificial Intelligence, and

Natural Language Processing from a layperson’s perspective

• Manipulating very large and specialized data sets such as geographic data

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

Department of State Programmatic Data Events

Duration: variable, not to exceed 1 day

Department of State Programmatic Data Events are designed to advance data literacy throughout the department. Such events may include, but are not limited to, hackathons, topic-specific presentations and workshops, and other events which advance the Department’s understanding of and competency with data literacy.

Employees who attend such events will have of the following performance outcomes:

• Increased awareness of data literacy and its importance to State Department strategic objectives and operations

• Increased competency and facility with core data analytics, visualization or storytelling skills

• Increased awareness of industry trends and best practices

Department of State Programmatic Data Events will be developed and offered on an as-needed basis to build upon and deepen State Department data competency. FSI will share a topic, specific objectives (if applicable) and event times and dates and ask any awardee to provide a quote to include a proposed workshop itinerary, proposed instructor, and quoted price to support the topic selected by SAIT. DOS shall review all quotes received by awardee(s) for the specific proposed data event and select the quote/awardee that aligns best with data event objectives considering the proposed itinerary, instructor, and quoted price.

Course Delivery Courses for all these audiences should consider a variety of delivery methods. In general, classes will be as a default offered as courses for audiences in physical spaces at Department facilities. Physical courses may be as short as a two-hour or three-hour seminar covering a very narrow topic, as long as a 40 hour course over 5 consecutive days on a more extensive topic, or somewhere in between.

However, given that many people are now working from home or social distancing when they venture into public spaces, there is also a need to consider offering courses in a virtual format.

Versions of the courses designed for virtual delivery may take a variety of formats based on the amount of content, the nature of the content, the types of interactions the content affords, and other considerations based on the content itself and the needs of the prospective audiences. Possible delivery methods may be: entirely synchronous online delivery, distributed over several days; blended delivery distributed over several days or weeks that combines regular synchronous sessions with videos, independent activities and/or group activities; or completely asynchronous delivery that combines playlists of microlearning videos with independent study activities and asynchronous mentor support. Other blends of teaching methods will also be considered.

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

The audience for any online or blended course may be located at any Department facility around the world, therefore instructors may need to teach synchronous sessions on schedules that conform to various time zones around the world.

Performance Requirements

1. Curriculum Design: The contractor will be tasked with designing and developing the content curriculum. Content will be designed for multiple delivery methods. Courses will have a delivery modality of in-person classroom courses consisting of approximately 12 students and a seat time of two hours or more but less than 40 hours. Additionally, course design should also consider virtual delivery. Possible delivery methods may include delivering courses entirely through synchronous online delivery, distributed over several days; blended delivery distributed over several days or weeks that combines regular synchronous sessions with videos and activities; or completely asynchronous delivery that combines videos with independent study activities and asynchronous mentor support.

Other blends of teaching methods will also be considered.

2. Delivery: Contractors may be the party to deliver the content, though options where the contractor sells or licenses the content for in-house teaching by SAIT staff will also be considered. In-person courses will be delivered at a Department facility. Most classroom courses will be taught in the Washington, D.C. locale, though there may be opportunities for teaching at other Department facilities around the world. For virtual delivery, contractor instructors may need to teach synchronous sessions on schedules designed for time zones other than the one in which they reside.

3. Collaborative Meetings: The contractor will participate in collaborative meetings with FSI staff to review and refine curriculum content and delivery plans. The content of meetings includes refining design plans, integrating workshop feedback, and guiding learning scenarios that align with the context of FSI.

4. Technical Specifications: When applicable, students will have access to PCs with Windows 10 operating system, an internet connection, and Microsoft Office 365 and Microsoft Power BI. If a course requires other specialized analytics tools other than Office 365 and Microsoft Power BI, that must be specified and cleared during the course design process, and will be considered on a case-by-case basis. Students will not be permitted to download new software or perform software updates on existing software.

5. When working on premises, the contracted instructor(s) will be able to use the audio/visual equipment in the classroom, include large-format and/or interactive displays.

In some cases, the contracted instructor may be given access to an unclassified Department computer, but unless otherwise specified, must bring their own laptop and audio-visual connectors to the training.

Desired Performance Outcomes

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

The following list of performance outcomes is not meant to be exhaustive but provide examples to be refined based on vendor capabilities and responses. These outcome examples could also very well be delivered across a series of distinct workshops (for example separate intermediate-level workshops, advanced-level workshops as well as separate workshops for producers and consumers).

Producers of Data Analysis

• Understand foundational data literacy principles

• Collect, clean and check the accuracy of data from a variety of sources and at variety of dataset sizes

• Consider data bias and potential data flaws in the data

• Prepare a range of dataset types (numerical, text, geographic data sets) for analysis at varying levels of complexity (intermediate to advanced)

• Perform data analysis at various ranges of competencies (intermediate to advanced), using tools such as Microsoft Excel and Microsoft Power BI, and other analytical tools where appropriate

• Identify and develop appropriate data visualization techniques, data storytelling, and solutions based on content, audience, and method of delivery to intended audience

Consumers of Data Analysis

• Understand analytical and statistical thought processes, discipline, and concepts

• Understand and consider issues inherent to producing data analysis: including data bias, data security, and data management and governance principles

• Identify effective and flawed features of data analysis results and visualizations

• Develop effective arguments, presentations, cases based on available data analysis

• Develop recommendations for incorporating data analysis into Department strategic planning processes and documentation supporting Department policy formulation

Operating Constraints While the students will have access to the internet, they will not be able to download unapproved software, or update existing software, for use during delivery of the workshops.

All software must be approved in advance.

Place of Performance/Delivery

This document is a draft PWS intended for use as part of a Request for Information (RFI). It has been

The contractor will deliver instruction to employees in a classroom or lecture hall at a Department of State facility, usually, the Department of State’s training campus: 4000 Arlington Blvd. Arlington, VA 22204*.

Department of State George P. Shultz National Foreign Affairs Training Center Shultz Center FSI Washington, DC 20522-4201

*See Questions section – if the contractor or vendor has a training facility in the metropolitan area, we will consider the feasibility of conducting the training at the vendor’s facility.

Government-Furnished Property When working on premises, the contracted instructor(s) will be able to use the audio/visual equipment in the classroom, include large-format and/or interactive displays. In some cases, the contracted instructor may be given access to an unclassified Department computer, but unless otherwise specified, must bring their own laptop and audio-visual connectors to the training.

Security Clearance Requirements None.

Anticipated Period of Performance Beginning in FY20 Q4