280-EIPL-1112 Questions and Answers - Final.pdf

No. Question Answer 1 Who is the incumbent (if any) related to the sources sought? There is no incumbent for this requirement.

2 What is the incument's current project/job/contract number? There is no incumbent for this requirement.

Is the agency open to placing the prospective requirement into the 8(a) program?

The USPTO is still conducting market research. This Patent and Trademark Acquistion Guide (PTAG) 6.1.1. is part of that market research. As such, USPTO has not yet determined its final acquisition strategy, to include business size and socio‐economic considerations (e.g. small business set‐aside, 8(a), etc.). The USPTO has not yet ruled out any strategy. Based on the results of this first phase of market research, the USPTO may determine that a small business set‐aside strategy may be employed before moving into the next phase of market research.

Is there going to be a small business set‐aside or will this be bid as full and open? The USPTO is still conducting market research and has not yet determined its final acquisition strategy, to include business size and socio‐economic considerations (e.g. small business set‐ aside, 8(a), etc.). The USPTO has not yet ruled out any strategy. Based on the results of this first phase of market research, the USPTO may determine that a small business set‐aside strategy may be employed before moving into the next phase of market research.

What is USPTO's anticipated budget for this requirement? The two‐year budget for this requirement is anticipated to be below the simplified acquisition threshold ($250K).

Could you please confirm whether vendors will receive an acknowledgement of receipt from PTO upon submitting RFI Responses on April 18, 2024?

Yes, vendors will receive an acknowledgement receipt upon submitting its PTAG 6.1.1. RFI response.

7 Please indicate anticipated/expected date for PTO’s response to Questions. All questions will be answered via RFI amendment.

Please share information about how responses to the RFI will be evaluated and what criteria will be used to assess the vendor’s submission for promotion from Stage One to Stage Two?

The purpose of this RFI is to conduct market research in accordance with PTAG 6.1.1. This is not a formal source selection, therefore, evaluation criteria are not required and will not be developed for this market research endeavor. RFI submissions will be assessed to determine which vendor(s) is/are the most likely to successfully meet the agency’s needs and are thereby eligible to participate in an alternative competition under PTAG 6.1.1..

1. Please indicate what is the anticipated timeline for reviewing the RFI responses and moving forward with any subsequent actions?

a. Anticipated Start Date for Stage Two

b. Anticipated Start Date for Stage Three

c. Anticipated Start Date for Stage Four

As USPTO is in the early phase of market research, it is difficult to accurately define the requested timelines without knowing the full level of interest from industry. As such, the below dates are provided as estimates only for planning purposes:

a. Anticipated Start Date for Stage Two: June/July 2024

b. Anticipated Start Date for Stage Three: August/September 2024

c. Anticipated Start Date for Stage Four: September/October 2024 (Issuance of Solicitation(s))

Does a contract currently exist to support the PTO and/or OCIO for automation services (design, development, deployment, CI/CD)? If so, please provide the

a. Contract Number

b. Contract Name

c. Incumbent

d. Commercial Off‐The‐Shelf (COTS) product being used?

USPTO has many contracts that involve automation in some way, from chaos testing, to RPA, to CI/CD pipelines, etc. However, as those contracts do not directly relate to this requirement, the USPTO does not intend to provide specific contract information for each automation related requirement.

Can the USPTO provide more detailed criteria on how vendors will be evaluated in Stage One of the market research process? Specifically, what evidence or characteristics will make a vendor stand out as likely to meet the agency's needs?

The purpose of this RFI is to conduct market research under USPTOs Patent and Trademark Office Efficiency Act's (PTOEA) special authorities in accordance with PTAG 6.1.1. This is not a formal source selection, nor is is a solicitation, therefore, evaluation criteria are not required and have not been developed for this market research endeavor. RFI submissions will be assessed to determine which vendor(s) is/are the most likely to successfully meet the agency’s needs and are thereby eligible to participate in an alternative competition.

Could the USPTO provide insight into how each requirement will be weighted in the evaluation and decision‐making process? Knowing whether certain requirements are considered critical, highly desirable, or just nice‐to‐have would enable vendors to tailor their responses and demonstrations more effectively, ensuring that they address the USPTO's most pressing needs with the appropriate level of detail and emphasis.

All requirements contained within the RFI are considered to be important to the future success of the anticipated Automation and Orchestration Platform. RFI submissions will be assessed to determine which vendor(s) is/are the most likely to successfully meet the agency’s needs and are thereby eligible to participate in an alternative competition.

13 What are your preferred development languages and tooling (e.g., Angular, React, Node.JS)?

Preferred developmental languages and tooling are not relevant. Although one of the requirements of the RFI is the ability to write custom code, a vast majority of what the USPTO wants is a no‐code solution.

Will solutions be hosted on USPTO infrastructure? If yes, what are your preferred platforms (e.g., AWS, Azure, GCP, OpenShift, Docker, Kubernetes)?

Solutions will be either vendor‐hosted/SaaS or something USPTO can deploy in its AWS UACS environment.

15 What applications will integrate with the automation and orchestration layer? The objective is to easily integrate with ANY web application.

What is the estimated volume of process instances initiated daily (10/day, 100/day, 1k/day, 10k/day)?

There is no estimated volume.

17 How do you use rules within your workflows to enable straight through processing?

Due to lack of context, the USPTO does not intend to provide a response at this time. The PTAG market research RFI hopefully will be able to inform USPTO's future response.

What types of orchestration endpoints (e.g., APIs, bots, business rules) are you using to integrate with legacy systems?

USPTO is using all three but, will be focusing mostly on Application Programming Interfaces (APIs) moving forward.

How many endpoints do you estimate will be connected to the average application to enable straight through processing?

Due to lack of context, the USPTO does not intend to provide a response at this time. The PTAG market research RFI hopefully will be able to inform USPTO's future response.

20 What is the percentage of human‐driven processes vs system‐driven processes?

USPTO does not have a percentage, but the balance is very heavily weighted toward system‐ driven processes.

21 How do you commonly build your human interfaces? Not applicable.

22 What is the general complexity of your workflows (5 steps vs 50 steps)? This will vary per use‐case and can be anywhere from 10 to 100+ steps.

23 What is the general complexity of your forms (10 fields vs 100 fields)? Not applicable.

24 What type of data are you sourcing internally? A wide variety of user and business data will be sourced internally.

25 What type of data are you sourcing externally? A wide variety of user and business data will be sourced externally.

26 How do you commonly initiate workflows (user initiated, system initiated, rule initiated)?

Workflows in this platform will mostly be system initiated, either from the platform or other applications.

27 What are reasons why is it important to test only part of the process application at one time?

One of many reasons we want to test individual or sub‐parts of an automation is for usability.

Running 10 steps out of 100 at a time saves time and makes it easier to build, test, debug, and maintain an automation workflow.

28 What are the top three challenges you are trying to resolve?

Two big challenges are 1) Easily building automations/workflows that integrate with a wide variety of applications that are both common/external (Microsoft Teams/SharePoint, Service‐ Now, Gitlab) and internal (Patent center/search, Trademark Center, etc). 2) Automating complex business processes that involve a lot of back and forth, such as multiple levels of approval and authority to operate (ATO).

29 What are the savings from automating one instance of a workflow? Savings vary widely depending on use‐case.

30 What are your success metrics?

In terms of future success of the platform, the USPTO seeks: 1. Time saved (amount of time saved by a workflow multiplied by how many times it runs); 2. Positive return on investment (time saved per year * hourly rate is ideally below cost of the platform); and 3. Reduced manual/human error, increased consistency, save time, scale/automate processes.

RFI submissions will be assessed to determine which vendor(s) is/are the most likely to successfully meet the agency’s needs and are thereby eligible to participate in an alternative competition. As this is a PTAG 6.1.1. market research effort, these should not be considerd as evaluation factors, only projection of measurements of future success.

Could you specify the approximate number of named users expected to access the USPTO automation platform?

USPTO is seeking a platform that provides read‐only user access for the entire agency (all government full time employees + all contractor full time emploiyees, currently around 16,000 people) and 200 builders.

What specific roles will the users of the automation platform occupy? Are they primarily involved in security operations, or will they be engaged in general business automation processes?

Users and use‐cases from all across the agency, from security operations, to fraud operations, to cloud/infrastructure/pipelines, to vulnerability management, and multiple governance efforts.

Is there a need for the platform to be supportive of No Code development patterns like "citizen development"?

Yes, one aspect we will be evaluating is "ease of use" which does include the ability for non‐ developers to build workflows.

What are the specific pain points that these are trying to address, is there an existing platform/system where this has proven difficult?

There is not an specific existing platform/system where this has proven difficult. However, in general the USPTO is seeking to reduce manual effort spent on repetitive tasks, reduce error, increase consistency.

Can you provide the list of key COTS and custom applications and systems in operation at PTO, along with the key technologies for each of them. At minimum, front end (UI), database, middleware, workflow related technologies.

No list of COTS or customer applications will be provided, as one of the major requirements for the platform is to be able to easily integrate with any system’s Application Programming Interface (API). As we will be creating workflows that automate and orchestrate via APIs, no mention of front‐end, database, et al are needed.

Can you provide an overview of the Identity and Access management systems and tools used for your current applications and systems. Do you have a corporate level LDAP with organizational hierarchy defined?

USPTO has corporate level Lightweight Directory Access Protocol (LDAP) with organizational heirarchy defined (Azure Active Directory) and both are internal and external applications use Okta for single sign‐on.

Can you provide information about any single‐sign‐on authentication methods used for your current applications?

USPTO uses Okta and Active Directory for Single Sign‐on.

Are your business processes or workflows documented somewhere? If not, how do you envision providing these requirements ? Will you have stakeholders available to attend process discovery workshops to flesh out the details of your current and future business processes.

Some processes are documented and some are not. We do not plan to provide any in‐depth requirements, as workflows will be built by existing staff. Stakeholders are only available to existing staff to meet with if needed.

Is there a need or thinking around establishing a Center of Excellence or similar governing body to manage the new platform?

USPTO is not considering the establishment of a Center of Excellence or similar government body to manage the new platform.

Would you accept a managed services platform hosted on a FedRAMP compliant cloud infrastructure (such as AWS GovCloud) for the AOP solution?

Yes.

Are there IT processes and procedures that a contractor implementing the AOP system will need to follow? If so, can you share some information about these processes and procudures?

USPTO does not foresee the need for contractor personnel performance under this requirment, as a the desired solution is a product, not implementors.

Can you share the security and privacy standards any new system needs to meet and the key steps to get an autorization to operate a new system?

The new system should not require anything outside of any standard ATO process. If the vendor is hosting the application in the cloud, it should be FedRAMP certified

Can you clarify the following requirement is section 2 of the RFI Request ‐ "platform that can integrate easily with any Application Programming Interface (API), and not just those in which an integration has been pre‐built." By definition, for APIs such as SOAP or REST to work there is a publisher and a consumer of the API endpoint. The published API specifications can be standard based and not tailored for any specific consumer. Is it correct to assume that the applications to integrate with the AOP provide such standard API endpoints that may not be tailored for any specific consumers ? or are you asking that even if the existing application does not expose any APIs, the AOP should still be able to integrate with it ?

If an API exists, whether it is tailored/created for consumer/consumption or not, the API should be able to easily consume/integrate/automate using this product without any custom development.

Would you accept a capacity based model for the labor, where the pricing is provided for a cetain number of Agile Teams for a ceratin duration? The functional scope for each team for each sprint will be determined using an Agile product backlog process.

No, USPTO anticiaptes the purchase of a product and not labor.

What is the total number of systems/applications that the USPTO is looking to integrate with this platform?

USPTO is looking to integrate 200+ systems.

46 What is the maximum anticipated steps of the largest workflow? The maximum anticipated steps in the largest workflow is 300.

47 How many workflows does the agency anticipate to run per day/month/year? This has not yet been determined.

Are there peak periods of processing? If yes what are the anticipated peak numbers of workflows run?

No, there are no peak periods of processing.

49 Is there a master data hub or golden record for data currently in place? No, there is not a master data hub or golden record for data currently in place.

50 How many users will need access to this system?

USPTO is seeking a platform that provides read‐only user access for the entire agency (all government full time employees + all contractor full time emploiyees, currently around 16,000 people) and 200 builders.

51 What is the average number of steps per workflow? Number of steps per workflow varies per use‐case and be anywhere from 10 to 100+ steps.

Desired Outcome: What strategic outcome(s) will the USPTO be able to achieve by way of an Automation and Orchestration solution that would positively impact the Mission of the USPTO?

The USPTO does not intend to provide a response at this time.The PTAG market research RFI hopefully will be able to inform USPTO's future response.

Demonstration Expectations: During Stage Two, what are the specific challenges or scenarios the USPTO would like vendors to address in their live environment product demonstrations?

The USPTO does not intend to provide a response at this time. The PTAG market research RFI hopefully will be able to inform USPTO's future response.

Evaluation Criteria for Demonstrations: What specific functionalities, capabilities, or performance metrics will the USPTO focus on during the product demonstrations to assess each solution's effectiveness?

The USPTO does not intend to provide a response at this time. The PTAG market research RFI hopefully will be able to inform USPTO's future response.

Integration Specifics: Could the USPTO elaborate on the types of custom applications (e.g., technologies, platforms) and the specific challenges encountered with current integrations? This detail would help vendors understand the depth of integration capabilities required.

The biggest challenges are less technical in nature. They arise as workflows scale up in complexity and number of steps, resulting in technical debt and solutions that are slow to build, hard to maintain, and difficult to debug.

Scale of Automation: Can the USPTO provide examples of the scale and complexity of the automations envisioned, especially those involving fraud operations, cyber security operations, and governance/monitoring?

a. Annual Transaction Volumes: Could the USPTO provide an estimate of the annual number of transactions that the automation and orchestration platform is expected to handle?

(Understanding the volume of transactions, including peaks and averages, would help in assessing the required scalability, reliability, and performance capabilities of the proposed solutions.)

USPTO estimates running between 20,000 and 30,000 steps (actions) per day. Since not all steps are transactions, this means between 7,300,000 and 10,950,000 transactions per year. Peak information is not available.

Integration Needs with Non‐API Systems and RPA Usage: Given the variety of systems and custom applications in use at the USPTO, some of which may not offer API access for integration, does the USPTO anticipate the need for Robotic Process Automation (RPA) to automate interactions with these systems? ServiceNow provides RPA capabilities that can automate tasks with systems by mimicking user interactions, thus not requiring an API. If RPA integration is needed, could the USPTO provide insights into how they envision the use of attended robots (requiring human interaction) versus unattended robots (operating autonomously)? Additionally, an estimate of the number of each type of robot needed would be invaluable for designing an automation platform that is well‐aligned with the agency's operational workflows and human resource strategies.

USPTO is currently using UI Path for both attended and unattended robots. This request is not looking for another Robotic Process Automation tool.

Testing Integration with Non‐Public Applications and APIs: Given the USPTO's likely use of a mix of Commercial‐Off‐The‐Shelf (COTS) products and custom applications, some of which may involve non‐publicly available APIs or are hosted in secure environments, will the USPTO require the ability to test integrations with these non‐public applications and APIs as part of the evaluation process? If so, could the USPTO provide guidelines or requirements for how vendors can securely access these systems for the purpose of demonstrating integration capabilities and ensuring compatibility with the ServiceNow platform?

USPTO will only be testing integrations/systems that are available in our lab (development) environment and/or integrations with applications that are accessible without any additional network configurations. USPTO will work with vendors to test their product in our lab environment, whether that is direct deployment or deployment of a tunnel/reverse proxy

Interest in Replatforming Legacy Applications: Recognizing the USPTO is using custom applications and the need for seamless integration with a variety of systems, would the USPTO be interested in evaluating how a workflow platform like ServiceNow could assist in replatforming custom applications? This approach could not only modernize and streamline operations but also enhance the efficiency and scalability of the USPTO's digital infrastructure. Understanding the USPTO's openness to such replatforming initiatives would aid vendors in showcasing relevant capabilities and success stories related to legacy application modernization.

This is not in the scope of this requirement.

Current Implementations: Are there current Automation platforms in use at USPTO? If so, could you briefly describe the current use cases, the scale, and what platforms are being leveraged?

USPTO uses a wide variety of tools and custom code to automate various use‐cases. Two examples are UI Path for robotic process automation and GitLab for development pipeline automations.

Can the USPTO clarify what specific problem(s) are needing to be addressed and solved by way of an Automation and Orchestration solution?

Two big challenges are 1) Easily building automations/workflows that integrate a wide variety of applications that are both common/external (Microsoft Teams/SharePoint, Service‐Now, Gitlab) and internal (Patent center/search, Trademark Center, etc). 2) Automating complex business processes that involve a lot of back and forth, such as multiple levels of approval and authority to operate (ATO).

Can you provide a high level overview of the cloud and on‐prem infrastructure used to host your systems?

USPTO has a mixture of on‐premise infrastructure (Alexandria VA + Manassass VA) and cloud infrastructure (mostly Amazon, but some presence in Azure and Google).

Can you provide an overview of network connectivity between your cloud platforms, on‐premise applications etc?

USPTO uses site‐to‐site tunnels, virtual private networks, virtual private clouds, and secure access service edge.

Can you provide the list of technologies currently supporting the existing PTO systems and applications and require integration.

Not applicable.

Can you provide an estimate of the following metrics needed to estimate the level of effort for such AOP solution ‐ number of businjess processes (workflows), average number of steps per process, number of user roles, number of users, number of user interface screens, number of application/system integrations etc.

Each workflow will have between 10 and 100+ steps. USPTO is looking for unlimited read‐only users (all government full time employees + all contractor full time emploiyees) and around ~200 builders. USPTO is looking at 200+ integrations. USPTO does not have plans to use this platform for user interface screens.

Can the government elaborate or provide more information about both the process and expected timelines for the Stage 2 live environment product demonstrations to assist vendors with preparations?

As USPTO is in the early phase of market research, it is difficult to accurately define the requested timelines without knowing the full level of interest from industry. As such, the below dates are provided as estimates only for planning purposes:

a. Anticipated Start Date for Stage Two: June/July 2024

b. Anticipated Start Date for Stage Three: August/September 2024

c. Anticipated Start Date for Stage Four: September/October 2024 (Issuance of Solicitation(s))

Does the USPTO require the automation platform to include capabilities for security orchestration and incident response? If so, could you detail the types of security events the platform should respond to and the expected incident response activities?

Yes. There are a wide variery of security events, a few examples being phishing, GuardDuty alerts, and Okta alerts (such as impossible travel).

Is there a requirement for automating the collection, analysis, and dissemination of cybersecurity threat intelligence at the USPTO? Could you provide specifics on the scope and nature of the threat intelligence automation needed?

No, there is not a requirement for automating the collection, analysis, and dissemination of cybersecurity threat intelligence at the USPTO.

Is it acceptable for the manufacturer and chosen partner/reseller to submit one combined RFI response? Or is it required for the manufacturer and partner/reseller to submit separate RFI responses?

Per section 5.2h of RFI amendment 02, a single submission for a teaming arrangement between a manufacturer and a reseller is acceptable, however, please be sure to follow updated submission instructions.

Can you share additional information on the types of API endpoints that USPTO envisions needing to connect to. For example, beyond REST, what other types of APIs are potentially in‐scope?

Would they be only published APIs or also private or proprietary?

Almost all Application Programming Interfaces (APIs) used will be Representational State Transfer (REST) and will be a mix of published (majority) and private/unpublished (minority).

71 What order of magnitude of APIs would USPTO envision connecting to? 10s, 100s, 1000s? USPTO estimates using between 10 to 100 different APIs per workflow.

72 Is USPTO only looking for a particular deployment option? E.g. On‐prem, IaaS, SaaS? No, USPTO is not seeking a particular deployment option.

Is USPTO looking for a user interface for case management or other end user interactions in addition to workflow orchestration?

No, USPTO is not seeking a user interface for case management or other end user interactions in addition to workflow orchestration.

In terms of reducing manual effort, is USPTO referring to / interested in automating decisions in the workflow?

Yes.

In terms of reducing manual effort, is USPTO referring to automating, but otherwise leaving unchanged, existing manual processes that users may have to perform interacting with computer systems? i.e. Copy pasting, clicking through screens in multiple systems, etc.?

No. Any new workflows created, or existing workflows migrated, will focus on integration and automation using Application Programming Interfaces (APIs), and not mimicing human interaction with a system.

With regards to change management, in addition to the basic capabilities outlined, is USPTO expecting the solution to fulfill a full‐fledged version control CI/CD system or to integrate with an existing USPTO solution in this space?

USPTO expects the product to have, at the very least, a change/version control system for workflows as part of the product.

Can you give an example of fraud operations or cyber security operations workflows, etc. that the USPTO has in mind?

An example of of cyber security operations is automatically responding to alerts generated by various tools such as Okta and Amazon GuardDuty. An example of fraud operations is responding to alerts of possible fraudulent activity in our public‐facing/business systems.