3 - 18F FEC Lotus Notes Technical Assessment - Final Report.pdf

18F + FEC

Technical Assessment Conclusions Evaluating options for replacing Lotus Notes

Prepared by Annalee Flower Horne, Yoz Grahame, and Angela Colter

September 2017

Table of Contents Introduction 3

What we’ve been asked to do 3 Executive summary 4

Findings 6

Software Issues 6 Outdated technology 6 Lack of continuous user interface improvements 7 Pain points for the OCIO 7

Development process issues 8 No regular, common process for app development 8 User needs are not driving development 8 Most documentation is either poor or nonexistent 9

Organizational issues 10 The Infrastructure Division is forced to work reactively 10 Contractors continue to do good work without long-term strategy 10 Lack of portfolio management capacity causes overspend and under-utilization 11 OCIO needs more and better communication with the wider organization 11

Recommendations 12

Agile guide 12

1. Avoid tempting anti-solutions 13

2. Understand the full range of user needs 14

3. Build cross-functional teams to reduce friction 14

4. Build an agile process 18

Acquisition guide 23

1. Use what you have 23

2. Procure teaching engagements 24

Migration guide 25

1. Tactics for app migration 25

2. Working with users to replace workflow apps 32

3. Individual application migration recommendations 35

App inventory 47

FEC System Access (FSA) 47 Emergency Contact DB 50 Enforcement Priority System (EPS) 52 Invoice Tracking Program 53 Document Voting System, DVS Staging 56 Certification 59 Assignment Tracking 60

Additional information 64

Recommended reading for agile process development 64 Acknowledgements 64 About 18F 64

Introduction The Federal Election Commission (FEC) OCIO currently runs nine custom Lotus Notes applications in various areas of the organization. The OCIO considers these applications costly and increasingly difficult to maintain, and is interested in finding replacement options and a path toward migrating users as soon as possible. To support this project, the FEC OCIO partnered with 18F to examine the use and usability of the Lotus Notes application system. This report is the culmination of those efforts.

What we’ve been asked to do

In the initial Statement of Work, we were asked to examine the nine Lotus Notes apps, interview FEC staff to understand each app’s use cases and core competencies, and collect feedback on the general usability of the system. Our goal was to define a strategic long term solution with recommendations concerning how to migrate off of Lotus Notes.

On August 23rd, we presented stakeholders with our initial findings and recommendations, and provided the following options for where we should focus our remaining efforts and structure the final report:

● Identify hidden, high-risk use cases by surveying all app users

● Provide tactical technical recommendations for migrating each app

● Provide strategic organizational recommendations for executing app replacement

This report is structured around the FEC’s decision to focus on organizational strategy as the top priority, with technical tactics second.

Executive summary

In the time since the Lotus Notes applications were created the needs of users and IT staff have changed, as has the technological landscape. Unfortunately, the apps have not been sufficiently updated to meet these changed needs, despite the continual expenditure on app maintenance. As a result, there’s a gap between what the apps currently are and what they should be, and this gap has widened every year. As the gap widens, so does the cost to fix it.

As we researched the apps and their histories, we found that this gap is about more than just the age of Lotus Notes. It’s about the need for a user centered product development process that can meet user requirements. Thanks to discussions with OCIO staff, we traced these outcomes back to common organizational patterns. Most of these issues are common across IT departments everywhere, whether in government or private enterprise. They happen when support organizations that were created to deal with a certain type of work find themselves having to do something very different. They are in no way unique to the FEC, and individual staffers are not to blame.

When considering the high cost of maintaining the current Lotus apps, the OCIO primarily considers the easily-calculated expenditures on licenses, servers, and contractors. However, there are wider costs and hindrances that are also known, but less easily calculated:

● Helpdesk requests due to user confusion, password resets, and other problems

● Poor relationships between IT and other divisions, due to difficulty meeting their needs

● Contribution to OCIO’s existing portfolio management issues

● Time wasted by users dealing with inefficiencies or bugs in the software

So, the question is: Given this mixture of different costs for maintaining the existing apps, can they be replaced cheaply and quickly?

The bad news: Whatever the outcome, the FEC OCIO is either going to incur significant expenditure or a dangerous level of risk. We have found no quick or cheap fixes without a higher cost elsewhere. A hurried set of app purchases without user involvement may result in apps that users refuse to take. Attempting to reimplement all the apps as currently designed may take far too long, increase maintenance load, and do nothing to fit the apps to the changing nature of the problems they address.

In order to avoid those outcomes, these apps don’t just need new technology; they need a product development process that guides the technology decisions. That process needs to be balanced for speed, cost, and quality. Unfortunately, these apps don’t currently have much development process at all, so that’s going to take some cost and time in itself.

The good news: We believe that there are cost-effective solutions for replacing these apps that also substantially improve the working situation for both users and OCIO staff. Obviously, there is potential for better software. But more significantly, there is potential for this improvement to continuously keep pace with changing needs. In doing so, it can reduce all of the costs and hindrances listed above. It can give IT staff more headroom, so they have the capacity to plan ahead rather than simply reacting.

And it can ensure that the OCIO does not find itself in the same situation in five to 10 years’ time.

Findings In researching the Lotus Notes applications and the user needs behind them, we discovered that the issues with the existing software are largely caused by issues with the development process. These process issues, in turn, stem from broader organizational challenges, including how contractors are managed, how work is planned, and how the wider organization relates to the Office of the CIO and the software they maintain.

Most of these findings will not be news to the OCIO. We identify them here because the problems in the legacy Lotus Notes apps, and the urgency around their replacement, are both symptoms of higher-level issues. Replacing the current legacy apps without addressing these issues will only lead to a repeating cycle of expensive research and replacement of systems that are not meeting user needs.

Software Issues

Outdated technology

As FEC’s Office of the CIO is already aware, key functions of the organization are running on outdated technology. Users we interviewed repeatedly expressed a desire for web-based and mobile-friendly applications. The age of the Notes app is also forcing users to rely on other outdated technology. For example, the certification app relies on ActiveX, which forces FEC users to rely on an old, insecure version of Microsoft Internet Explorer to complete their work.

The current system also has significant user experience shortcomings that put a strain on your help desk staff. Users often need assistance navigating the applications because the interfaces are not intuitive or regularly updated in response to user feedback. While FEC has standardized on PIV card + PIN to log into computers and O365, Lotus Notes requires a separate login, and infrequent users lose their credentials and need help desk support.

Progress towards modernization was inconsistent. Steps could have been taken to make the existing apps easier to port to more modern systems, but FEC’s Infrastructure team hasn’t had the capability to take advantage of these opportunities.

Lack of continuous user interface improvements

Many of the existing applications require manual work that could be automated.

Examples include regularly-scheduled tasks such as sending daily updates, copying data from one place to another, manual approval checks, and text entry fields that should be searchable drop-downs to reduce data entry errors.

Much of this is due to lack of integration with other systems, but it speaks to a lack of continuous, user-driven improvement to user interfaces. These errors have existed in the apps for years, but there’s no system in place to gather feedback from users and make improvements. Users have been trained to accept the systems as they are instead of regularly asking themselves and each other “how could this software better serve our needs?”

Pain points for the OCIO

The costs of these legacy apps in terms of servers, licenses, and contractors are easy to quantify. But they incur many hidden costs as well.

These apps eat up significant time for Help Desk staff. Setup and account creation require manual work from Help Desk staff, with tools they don’t otherwise use.

Because the interfaces are not intuitive and aren’t regularly improved based on user feedback, Help Desk staff are frequently fielding support requests from users who can’t work out how to navigate the applications.

There are also security and usage auditing concerns. Tools to track who is authorized to access the apps — and who is using that access — are inadequate or nonexistent.

When we requested a list of authorized users and their last logins so we could set up user interviews, Solutions By Design had to write custom code to provide us with that information — and even then, they were only to retrieve the last week of logins.

Infrastructure, development, and support staff for these applications are users, too.

The current applications are not serving their needs. Additionally, lifting these applications out of Lotus Notes and shifting them as they are to another platform or to standalone apps wouldn’t address the pain points the current apps create for OCIO staff.

Development process issues

No regular, common process for app development

Our interviews regularly surfaced that OCIO staff want to maintain and develop applications using agile methodology. Currently, the development process for the Lotus Notes apps does not appear to be following a straightforward methodology. For example, there is no issue tracking system, no standardized mechanism for gathering feedback, and no plan or roadmap for these apps. Communications between app owners and the developers is irregular, and depends mostly on given app owners and their vision for how to improve the apps. Improvement work, once started, is not always completed, as the developers frequently have to leave tasks undone when priorities shift suddenly.

User needs are not driving development

There’s no system in place to collect and act upon feedback from users, or center user needs in the development process.

One of the first things we heard when we started this engagement was “there’s nothing wrong with the software. The problem is the users.” But if users need extensive support to use software, the problem is always the software. Blaming users instead of seeing their questions as a form of feedback and an opportunity to improve the interface sets up an antagonistic relationship between users and IT staff, and blocks improvements that could save everyone — end users and support staff alike — a great deal of time and effort.

This communications breakdown also leads to maladaptive behavior from users.

Because they haven’t been engaged as partners in the development process, end users see the software as largely immutable. They have learned to think in terms of the existing workflow instead of the underlying task or problem, and tend to adapt their own working process to the software (including developing workarounds such as printing out items for their supervisor to approve because their supervisor was unable to log into the application, or using DVS to search for cases because the user couldn’t work out how to filter search results in ECM).

During interviews, we identified several possible changes to apps that the developers judged to be easy to implement and which resolved ongoing pain points for users, but these changes were never considered before. For example, for years, OCS staffers have been hitting buttons at 11am and 4pm each day to send out distribution emails, because they didn’t realize this process could be automated.

Users should understand that the software can be updated, and that they can ask for changes. More importantly, they should see that their requests are treated with respect, and result in improvements to the application. This builds trust that they won’t be judged or ignored when asking for changes. End users and IT staff alike need to understand that users shouldn’t be inconveniencing themselves to serve the application. The application should be serving them.

Most documentation is either poor or nonexistent

The primary motivator behind engaging 18F for this technical assessment was a lack of visibility into the architecture and dependencies of Lotus Notes apps. The fact that there is not already thorough technical documentation for these apps points to a larger problem with the development process. “Housekeeping” tasks like documentation are easy to push to the wayside in favor of pressing priorities, but when they’re put off indefinitely, it raises the cost of maintenance and replacement.

The most sorely needed documentation for these apps — and what we spent most of our time researching — is not specifically what they do, but why. Understanding the underlying user need and what the feature or application is meant to accomplish is essential to iterative, user-driven development. It is the north star that guides all development decisions, and assures that the team stays on course to serving the user, instead of serving the application itself.

Organizational issues

The Infrastructure Division is forced to work reactively

The issues with the development process bubble up into wider organizational issues around how FEC prioritizes development tasks.

The lack of a roadmap or product backlog hides the inherent trade-off in priority changes from stakeholders. As a result, priorities can change quickly and railroad current work, leaving tasks unfinished, and the state of those tasks undocumented.

Projects with low institutional priority get very little advanced planning.

The lack of forward planning and continuous improvement places a significant burden on the help desk. Infrequent Lotus Notes users need assistance navigating the applications and often need their login credentials reset.

In addition, because the Infrastructure Division is constantly reacting to competing priorities from stakeholders, they lack the capacity for ongoing user-centered development which would reduce their own workload and improve user experience.

Contractors continue to do good work without long-term strategy

Solutions By Design developed a working style that’s well-suited to Infrastructure’s reactive process for managing these applications. But while that might be meeting the immediate needs of the department, it’s not helping them meet the immediate or long-term needs of the users. They have no cadence, no medium or long-term targets, and there’s no assessment of their ongoing performance.

As a result, there is a steadily widening gap between where the apps are and where they should be to meet user and IT needs. The wider the gap grows, the greater the cost to the organization in terms of lost productivity, ongoing maintenance, and replacement costs.

Lack of portfolio management capacity causes overspend and under-utilization

Departments within FEC are procuring major and mission-critical software without any OCIO involvement. The OCIO doesn’t even learn of these purchases until the department comes to them seeking installation and ongoing support.

The OCIO is aware of this problem and working to bring all application procurement into their department, but it’s a huge concern that causes poor utilization of existing software and duplication across the organization, in addition to security concerns.

The organization has too many applications that could be better fit to their tasks if they were configured correctly and staff had adequate training, but there’s insufficient capacity in IT to research and configure these applications.

Coordinating these procurements through the OCIO will also facilitate better software sharing between departments, and reduce wasteful spending on software whose capabilities exist in other software the organization has already licensed.

OCIO needs more and better communication with the wider organization

The OCIO is currently operating over capacity. With more requests coming in than they have resources to respond, they’re forced into a reactive working style that leaves them no room to make long-term plans and prioritize tasks beyond “what’s currently on fire.”

A lack of visibility into the OCIO’s tasks and priorities makes it impossible for people in other departments to understand what their colleagues in the OCIO are working on and when their requests will be met. This leads departments to either try to strong-arm the OCIO into prioritizing their requests, or work around the office entirely by procuring systems without consulting the OCIO at all.

This came up in our Lotus Notes research when we learned that the department had procured software to get around using FSA, but the lack of transparency and continuous user involvement that are generating this pain point are also contributing to the widening gap between how the Lotus Notes apps work and what their users need.

Recommendations Ideally, we would be able to present you with a paper map of all the dangers and pitfalls of a Lotus Notes app migration. In doing so, we would lead you in the direction of a treasure chest with technical specifications and a plan to replace each app in turn.

Unfortunately, we aren’t able to do that. No consultant can give you a complete picture of the terrain, and no map can anticipate every pitfall you might encounter.

The good news is that your users will always understand your business needs — and the role they play in meeting them — better than we will. We can leverage that to give you something much more valuable than a plan that can’t possibly survive contact with reality: guidance on how to make your own map, and how to recover and adjust when the plan inevitably needs to change.

In the Agile Guide, our recommendations are structured around helping you build an agile process for your team that will help you gather and respond to user feedback so that your map stays up-to-date, and you retain the flexibility to change course to avoid blockers and pursue opportunities as they arise.

The recommendations in the Acquisition and Migration Guides are focused on helping you acquire the right software and transition to it smoothly. You’ll find further technical information on the existing apps in the App Inventory section.

Agile guide

1. Avoid tempting anti-solutions

In considering our recommendations, we also had to weigh the consequences of rushed or incomplete solutions. This led us to avoid what could appear to be tempting solutions and to explicitly recommend not pursuing them. Examples, with our reasons for avoiding them, are included below.

1.1 “First, gather the complete set of technical and feature requirements from the existing apps.”

This has been standard procedure for most of the history of government IT. It has caused too many project overspends, overruns and cancellations to count. While it is enticing to believe that you know, in great detail, all of the features and functions the system needs to do, history shows us that developing software this way does not lead to software that works for the users. These software applications exist to allow users to perform business processes, so if the applications don’t work for the user they don’t work for the business.

Iterating with user involvement not only leads to a superior product, it’s also faster and cheaper. This is because it targets the underlying business problem, not a set of static requirements. which may require far less work to solve than replicating the existing app.

We still recommend doing some initial research to uncover all the users and the first use cases for evaluation. Our work on this showed that there is a wider hidden set of use cases than we were able to discover, which leads to our recommendation of a user survey. (See “Understand the full range of user needs” further on in this section.)

1.2 “Give an external contractor the job of quickly developing replacement apps.”

It’s certainly tempting to do a quick replacement build and just get it done with.

However, if the contractors take requirements from the existing apps rather than the users, they’ll replicate most of the apps’ issues — and there’s a good chance they’ll deliver something which users dislike more than the existing apps.

That being said, having a user centered contracting team do the work with user involvement may be the best solution, as long as there is consistent management of the process in a structured, transparent way. In the Migration Guide below, we’ve identified some apps where this kind of ground-up rebuild actually is a feasible option.

However, that still leaves the question of the long-term costs of ownership. As we discussed during our kick-off presentation, we recommend that the apps stay

“continually open for development” rather than the traditional model of “complete, then maintain” — which is what’s been used around these Lotus Notes apps.

In trying to have a contractor do quick replacement builds, specific questions around long-term ownership arise: “If the development contract ends upon delivery, who’ll maintain these apps?” “If we keep the contractors on a retainer, how do we ensure that we receive appropriate value from the continual cost?” For example, Solutions By Design has been limited in the overall value to cost they can provide, mainly because the FEC-side owners haven’t had the process to identify and ask for that value. A quick replacement build executed by a contractor would experience the same pitfalls.

Additionally, some of these apps (such as Invoice Tracking) should be bought rather than built. And some apps should be built out on top of more suitable platforms — which FEC may already own — instead of being rebuilt from the ground up. These approaches will result in faster delivery of replacements which are also easier to maintain.

2. Understand the full range of user needs

Our initial research and interviews focused on the core users of the apps — those who use a given app multiple times a week. However, further research uncovered many users who use the apps more rarely and for a more limited set of tasks. This set of users represent a risk that we have not been able to quantify. We know these irregular users exist and removing the features they use may significantly impact their work.

We recommend surveying these users to better understand their needs and manage the risks associated with disrupting their workflows. Start with a wide and shallow survey: get the list of users for each app, then send them a SharePoint Survey to find out why and how they use the app. We strongly recommend keeping the survey questions short and simple to encourage a fast response; the aim is to gather just enough information to know whether a user should be interviewed in more depth.

Be aware that getting a useful response rate from surveys may require some cajoling.

Where possible, involve supervisors and department heads in encouraging staff to participate.

https://support.office.com/en-us/article/Plan-a-survey-215fedf4-0dcc-4454-8a4c-5600c06650bc

3. Build cross-functional teams to reduce friction

Each app will need a product owner, designers, developers, infrastructure/devops staff, and user support personnel. The idea behind cross-functional teams is to get all of those people together to work towards the common goal of serving FEC’s users, rather than siloing them off to worry about their individual objectives. The team members must have equal standing within the team and be in regular communication.

3.1 Move ownership to the Applications group

We recommend moving the Lotus Notes applications and their replacements under the Applications group.

The infrastructure group doesn’t have software development capabilities, nor the capacity, for long-term planning of applications work. The Applications group already has iterative, user-centered development practices, tools for development task tracking, and ownership of many existing custom apps and configurations. They’re well-suited to take over stewardship of these applications.

3.2 Bring each legacy app and its replacement under the same team

Involving the Solutions By Design contractors on the new app teams will significantly reduce the risk of a bad data migration. They have the best understanding of how the current applications and their database schemas work. They can provide insight into how users currently interact with their apps, either from experience or by analyzing data (for examples, see “Tactics for app migration” in the Migration Guide section).

This insight will be useful in making feature decisions.

3.3 Identify an empowered product owner for each app

A product owner is a single person who is responsible for managing a given application. They have the authority and responsibility to:

● Serve as the central decision maker and point of contact for their app

● Assign tasks and work elements

● Make business, product, and technical decisions

● Manage the app’s backlog of features and bugs

This product owner is ultimately responsible for how well the app meets the needs of its users, which is how any service should be evaluated. An app’s product owner should expect to devote at least fifty percent of their time to managing their application.

Their tasks will include:

● Recruiting users to work with the team

● Gathering requirements and feedback

● Turning this feedback into tasks, stories, and epics

● Prioritising the app’s backlog

● Working with the Applications Division to get features implemented and bugs fixed

● Verify delivery and quality of tasks

While it might be tempting to have the current app owner serve as the product owner, first consider whether the current app owner can accommodate the time demands of the role. The workload will be significantly higher and require much more communication. If the current app owner cannot accommodate the time required, identify another person to fill the product owner role. Each product owner should only responsible for a single app.

We recommend the following resources as an introduction to the product owner role and responsibilities.

● Agile Product Ownership in a Nutshell (15-minute video)

● A day in the life of an 18F product owner

● Why products built by large teams fail so often

3.4 Bring in product, agile, and UX expertise

All of these applications teams will need access to expertise in product management, agile development, and user experience design. We recommend bringing in a small team with these skills and making them available to all the applications teams.

The product expert should mentor new product owners and advise and assist with product ownership work.

https://18f.gsa.gov/2017/09/18/a-day-in-the-life-of-an-18f-product-owner/ https://www.youtube.com/watch?v=502ILHjX9EE https://blog.mojotech.com/why-products-built-by-large-teams-fail-so-often/

The agile expert should help the applications teams set up their agile processes, take part in team rituals such as backlog prioritization and retros, identify process improvements, and advise on agile process development.

The User Experience expert should mentor and advise the UX leads for each team, or scrub in as necessary to identify and fix design/user flow issues. You will likely need more than one User Experience designer for this work.

3.5 Put users at the center of the app development process

Your team’s ultimate goal is to help FEC staff execute the organization’s mission. As you build out applications, always remember that the software should serve the users.

Prioritize tasks — including features and bugfixes — based on what will most help the application’s users.

Remember that the app’s users include those in a variety of roles, some of whom may not use it on a daily basis. For example:

● Staff in other departments who don’t use the app directly, but who send or receive its data

● IT staff who administer or audit the app

● Occasional users who need to log in once every six months, and must re-learn the interface every time

● Legal and other regulatory staff who need to ensure that the app and its data meet guidelines

All of these users have needs for the app to meet.

As you build out your agile process (Section 3, below), make sure that you have clear and well-managed avenues for seeking and collecting feedback from your application’s users, and weight that feedback heavily when prioritizing tasks. An application bug that’s annoying a senior staffer but isn’t effecting the app’s usability should be deprioritized compared to a feature that will save a colleague three hours a week of work.

3.6 When engaging contractors, ensure that knowledge is being transferred

Given that FEC has an in-house engineering team, our primary recommendation is that contractors should be engaged for short term skills transfer rather than long term skills augmentation. Ensure that contractors pair with FEC engineering and infrastructure staff on implementation or configuration work. This is an excellent way to bring the required understanding in-house.

However, resource constraints may mean that there isn’t sufficient capacity in the team for that skills transfer. In this case, parceling the work out to contractors is a viable solution. The approach we recommend at 18F is known as modular contracting. These engagements should bring engineering and other needed skills into the team, to work closely with the product owner and users; the greater the distance, the more likely that the final delivery will not meet user needs. They must deliver sufficient information for the product team to manage the rest of the implementation with other engineers if needed.

For more information, see the Acquisitions Guide later in this report.

4. Build an agile process

4.1 Choose the tools you need

When talking about “agile,” it’s important to distinguish the toolbox of agile practices (such as daily standups, work-in-progress limits, and retrospectives) from Agile frameworks that use that toolbox to build specific workflows (such as the Scaled Agile Framework, Kanban, or Scrum).

The web team is using the Scaled Agile Framework (SAFe) to great effect. It’s tempting to copy their process wholesale, but you need to make sure your process is suiting the specific needs of your team. Every team needs to evolve an agile process that works for them; this is core to the “people over process” mantra from the Agile Manifesto. We recommend looking at the larger toolbox of agile methods and building an agile process iteratively, by bringing new methods in one by one and testing and refining the process as you go.

4.2 Schedule retrospectives

It may seem counterintuitive to begin at the end, but of all agile practices, retros are the most fundamental. They’re a vital mechanism for getting feedback, catching problems, and improving your process in an iterative manner. With retros in place, you’ll be in a https://modularcontracting.18f.gov/ better position to add other practices. Your team can try out those new practices knowing that you have a system in place for gathering feedback and making changes.

The simplest format for a retro is getting the team together to generate a list of things that went well and things that went poorly. It’s important to actually write these items down somewhere accessible, so that you can go back and review past retros to see if the same problems are coming up consistently. Many teams keep their notes from past retros in a single shared document.

For things that go poorly, the team should take some time to brainstorm how to prevent that problem from recurring. Some adverse situations cannot be avoided: for instance, if a government shutdown closes the office for three days, you can acknowledge in the retro that it slowed everyone down and then move on. But if an internal issue caused a disruption in regular work — such as a particular application having an all-hands-on-deck crisis and diverting staff from planned tasks — you can review why the crisis occurred, and what steps can be taken to prevent them in the future. Was a technical problem not caught in time? Did a senior stakeholder demand immediate work without regard for current priorities?

When a problem is identified that needs to be addressed, make sure the steps needed to resolve the problem are assigned to particular staffers as tasks, so they can be followed up as needed.

Choose a product owner who can take responsibility for running retros and otherwise managing your agile process. This product owner is responsible for making sure that agreed-upon practices are being followed, periodically reviewed, and improved or discarded as necessary.

4.3 Create a Kanban board

Difficulty with prioritization was a common pain point we heard from your team. IT staff feel like they’re jumping from task to task based on which priority has the person with the highest seniority “screaming the loudest” for it to get done. This creates stress on the IT team and impacts morale of other FEC staff, who don’t have a transparent view of why their requests are languishing while requests from their colleagues are jumping to the front of the queue.

Most agile frameworks include mechanisms to queue up and prioritize tasks. Some agile frameworks, such as SAFe, use sprints for prioritization: stakeholders get together for sprint planning and come to an agreement on the priorities for the length of the next sprint. The team has the certainty and stability of knowing what they’ll be working on for the duration of the sprint and doesn’t have to worry about being asked to switch gears mid-task, and the rest of the organization has a voice in how priorities are being set so that they don’t feel neglected.

Sprints are very effective for unified projects such as the website, but they’re often the wrong choice for teams that are juggling competing priorities and multiple projects.

The Kanban process uses a simple list of tasks that need to be completed, with clear and agreed-upon limits around how much work can be in the “in progress” list at any one time (see section 4.4 for more on these limits).

18F uses this process internally for acquiring software, getting ATOs, and hiring new staff. Stakeholders meet periodically to agree upon the order and priority of tasks in the backlog, and tasks are taken out of the backlog and completed in priority order. This gives the rest of the organization transparency into where they are in the queue so they can see what’s been prioritized ahead of them and why, while giving staff working on these tasks the room they need to focus on the task in front of them without worrying that they’ll be expected to drop everything and work on something else.

Given that FEC’s IT staff are juggling multiple projects and competing priorities for separate stakeholders and departments, we recommend adopting a Kanban approach over a sprint-based approach.

4.4 Set WIP limits

Work-In-Progress (WIP) limits are a common agile tool to manage competing priorities in a clear and transparent way. It’s common for teams to have a rolling backlog of tasks that need to be completed — especially teams like FEC’s IT and application staff, who consistently have more tasks to complete than they have hours in a day.

A WIP limit makes this bottleneck transparent, so that the team and other FEC staff have a better view of what’s in progress, and are better able to contend with the reality that tasks need to be prioritized and queued.

In sprint-based systems, the WIP limit is how many tasks are planned for a current sprint. In Kanban, the WIP limit is an explicit number of tasks that can be “in progress” at any one time. Before a new task can be started, an existing task must be completed.

The exact limit should be based on your team’s actual capacity, and should change only when capacity changes. For instance, if someone’s on maternity leave, the WIP limit should go down until they’re back. When you get a new staff member onboarded and fully trained, your capacity increases, and the WIP limit can increase.

4.5 Work in the open

Make sure that users and stakeholders have a clear view of your team’s in-progress and planned work. This may require creating a simplified version of your Kanban Board that’s easy for those outside the team to read at a glance.

Providing a clear picture of your IT teams’ ongoing work will help reduce demands for “squeaky wheel driven development.” When priorities are constantly shifting based on who’s complaining the loudest, staff learns that the only way to get prioritized (and to keep their requests from being preempted) is to complain loudly.

Giving other departments access to a prioritized list of IT department tasks will not only show them that their work is in the queue and how fast the queue is moving, but also make it clear that when they demand immediate service, they’re asking to preempt existing priorities: your team can’t generate extra time and resources out of thin air.

Knowing what your team is working on and why will make it easier for folks to wait their turn, instead of anxiously demanding immediate service. Transparency builds trust and improves inbound communication.

Make sure each product owner is an accessible point of contact for users. Users and other stakeholders should always be able to see what the current plans are for each app, including roadmaps, planned features, and known bugs. Users should have a clear and convenient way to provide feedback, and they should see that feedback incorporated into the app’s roadmap and priorities. That way, when it’s time to prioritize the department’s tasks, you can pull from an up-to-date list of each app’s prioritized tasks to make sure you’re investing your staff’s time where it will have the greatest impact.

4.6 Be selective about app work to include in each milestone

The Applications group doesn’t have the capacity to do continuous development on all these applications at once. Your users will be better served if developers focus their efforts instead of spreading themselves thin. To help them focus, organize the medium term work into milestones. Each milestone period should include a limited set of app projects to be worked on, with a target for that app to reach by the end of the milestone. The length of any given milestone is up to you, and they don’t have to be consistent, but you should be thinking in terms of several weeks.

Some projects will be blocked by other processes, such as contract negotiations, and can be deprioritized until those blockers are lifted. Some applications won’t receive non-urgent attention during a given milestone. (If users are significantly blocked by a problem, that’s urgent, and that work should be prioritized on the task board.) As long as lines of communication are kept open and the rest of the organization knows which apps are being prioritized and why, rotating through the apps so that they’re not all worked on during the same milestones is perfectly fine.

For applications that are prioritized and ‘shovel-ready,’ you can apply different disciplines to different apps: research and user experience can work on app A, then move on to app B while engineering picks up implementation work for app A. The engineering team can then pick up app B while user testing is done on app A. In that way, different staff can cycle through the applications in an assembly-line fashion that doesn’t leave anyone on the bench.

This explanation of milestones on the SAFe website explains the concept more thoroughly.

4.7 Compare notes with other agile teams at FEC

While your agile process needs to be built to suit the needs of your team, you do have an opportunity to compare notes with and learn from existing agile teams at FEC.

Review the agile software tactics mentioned in the e-Filing study: test automation, Test Driven Development, and automated deployment. In our experience, the initial effort to get started on these practices pays off quickly. Refactoring gets easier, major code changes go more smoothly, urgent bug fixes rear their heads less frequently. Most importantly, the time between users noting a problem and seeing a fix is greatly reduced.

Your goal is not to copy practices wholesale from other teams. Start small and adopt practices one at a time. Aim for gradual, steady improvement. Always remember that each team is different, and will find benefits or problems in different practices.

As you build out your own agile process, prioritize practices that will deliver the biggest gains for your team.

Acquisition guide

As you can see from our recommendations, many of the needs Lotus Notes apps are currently meeting could be better served with COTS software. Given how long acquisition timelines can take, we recommend pursuing a minimalist approach comprised of two considerations, both emphasizing modular contracting.

1. Use what you have

Our interviews with members of the IT support and apps team surfaced a pain point around software acquisitions. As one interviewee put it: “we buy expensive apps and then we don’t have the time to set them up properly. They have good features we’re not using.” There’s a high likelihood that you already have COTS software products within your ecosystem that can suitably replace the legacy Lotus Notes apps.

Considering these COTS products first could potentially save you time and money on acquisition, security and IT standards review, installation, and configuration. If you find https://www.fec.gov/about/reports-about-fec/agency-operations/e-filing-study-2016/how-move-forward-software-development-workflow-technical-road-map/ http://www.scaledagileframework.com/milestones/ a given application suitable for replacing functionality that currently lives in Lotus Notes, the next step is to determine whether your contract for the application support the scaling, in terms of license quantities and dollar amount, necessary to roll them out more widely. Depending on whether the application can meet your needs and whether its license can scale without difficulty, a new contracting action may be required.

Once you’ve chosen the software, changes to your software acquisition process can help you make better use of software you’re buying and free up development and support resources.

When acquiring complex software that requires configuration and training, include a sales engineer or other contractor engagement in the software contract. Have the sales engineer work with in-house staff on configuration and training. Set aside time with the sales engineer to discuss your current needs, so they can identify key features of the app that can help your staff.

If the app has useful features that aren’t immediately necessary to the transition, put those features in the product’s backlog and set aside a future time to review and train on the new feature after migration is complete.

2. Procure teaching engagements

For software like ServiceNow, which has extensive configurability, teaching engagements can train your applications and support staff on how to modify the software.

These are short-term contracts — roughly three to six months — with support personnel for the software you’re training on. The goal is knowledge transfer, and establishing best practices for your ongoing customization processes. The contractors should also assist with integrating the FEC’s Single-Sign On solution, without which users may need regular Help Desk assistance.

Whenever these contractors are customizing the software, they should be pairing with members of the applications team — side by side at the same computer. The goal isn’t for them to augment the software for you, but to teach your own staff how to customize the software, so that they can continue to make changes on an ongoing, iterative, user-driven basis.

ServiceNow is the perfect candidate for this type of teaching engagement. You should hire ServiceNow contractors who can teach you:

● How to set up new workflows

● How to modify workflows so you can iterate on them over time

● How to version control workflows using ServiceNow’s integrations with Git

● How to create automated tests for ServiceNow

Once a suitable COTS software product is identified and scalable, the focus should be on building up the technical capabilities of the existing Applications group rather than trying to replicate them with a new team. This team should also consider talking to candidate COTS software product(s) providers about training and configuration, and meet periodically with the vendor to talk about current needs and how the software can be configured to meet those needs.

Remember that even if an app is being bought, it still needs a team, including a product owner, contracting officer, key users, and a representative from Infrastructure/IT. They should work closely with a User Experience designer to identify user needs and efficient workflow options.

The General Services Administration’s Technology Transformation Service, Office of Acquisitions (also known as Acq Stack) can assist with setting up any acquisition effort for success.

Migration guide

1. Tactics for app migration

The replacement of an existing, heavily used app is a gradual process. A new app must be regularly tested by its intended users during development, while the app is incomplete. To get the most effective feedback, users should be working with real data.

Having an old and new app running side by side, both using and creating production data, can create major usability and data integrity challenges. Fortunately, there are tactics available which can make this easier.

1.1 Minimize user pain

Continual changes in user workflow, especially between multiple apps, can cause stress. This stress is passed on to colleagues, to user researchers, to development teams, and to Helpdesk staff. Stress makes it harder to gauge the effectiveness of replacement apps.

The users who are directly involved in the design of the new apps will be less impacted. They should already know what changes are coming, and why. It’s fine to give these users regular changes as part of the development process. In fact, they may be encouraged to see these regular changes happening, acknowledging the value of their involvement.

However, the other users have far less involvement. To them, many of these changes will be frustrating, and the most frustrating will be around managing two apps instead of one.

Given that the app replacement process relies on user cooperation and success, it’s worth putting in extra work to lessen the impact of app changes. As much as possible, ensure that users who aren’t involved in the design process are shielded from the problems of running two systems together. Keep them on the system they’re used to until it’s time for final migration. (See below for some technical strategies that make this easier.)

After the migration occurs, make sure to communicate changes through regular release notes and training, so that users who are not actively providing feedback and reviewing changes still know when things are changing and why.

1.2 Analyze user data to understand usage

When choosing existing features for implementation in a replacement app, there may be uncertainty around whether and how an existing feature is being used. In these cases, it’s may be helpful to analyze existing log data to look for evidence of usage. If there isn’t sufficient data for analysis, adding minimal logging code to the legacy app may provide more solid evidence.

1.3 Slice workflows for alpha/beta implementations

Most of the apps being replaced implement case workflows. A case is created by user in group A, then routed to a user in group B, then back to the original user in A, etc.

The nature of workflows means that there are multiple options for implementing and testing a workflow in the replacement app while the legacy app is still in use.

● Slice by workflow: This is the simplest and least granular. An entire single workflow will be implemented in the new app, and the legacy app will redirect users of that workflow to it.

● Slice by users: Most users in the case-creating group will continue to use the app, but a small subset of candidates will create their cases in the new app. The other groups of users will also be limited to candidate sets. Limiting users in this way makes it simpler to discuss and test the new flow with those users, and set up their user accounts.

● Slice by sub-workflow: Any contiguous group of steps within a workflow is, in itself, a workflow. Pick a section of a workflow to implement first, and decide how cases should flow between the new app and the old; it could be manual, with users copying the data over. Though that sounds time consuming, it’s useful for a quick test, and allows faster iteration. You can also combine this approach with slicing by user.

1.4 Maintain and share a single data translation document

Data to be transferred from the old to the new app…