220000000454.pdf
PDF 4 MB Posted
- Contract number
- MA220000000454
- Issued by
- Michigan
About this file
Summary
This document is a state and local contract indefinite delivery vehicle for Treasury Performance Testing Services executed between the State of Michigan, Department of Technology, Management and Budget (DTMB), and Foulk Consulting Services, Inc. The contract encompasses multiple change notices tracking amendments and modifications to the original agreement dated June 1, 2022. The primary agencies involved are DTMB and the Michigan Department of Treasury, with Foulk Consulting Services, Inc. located in Livonia, Michigan, serving as the contracted vendor.
The contract is for performance testing services on Michigan Treasury Online (MTO) and related Treasury applications including eServices, Siebel, and IVR systems. Services include load testing, scripting development and maintenance, performance analysis, and reporting using Tricentis NeoLoad testing tools. The initial contract value is $750,000 with an initial effective date of June 1, 2022, and expiration date of May 31, 2027. The contract includes five one-year option periods. Subsequent change notices have modified the contract value: Change Notice 1 added $32,880; Change Notice 2 deducted $32,880 but added $37,400; Change Notice 3 added $86,820; Change Notice 4 added $23,680; and Change Notice 7 and 8 reflect adjustments of $50,886. Payment is made on a satisfactory acceptance of deliverables basis within 45 days of properly completed invoices, submitted no more than monthly. Work is performed remotely, with no travel expenses reimbursed. The contract is issued under contract number MA220000000454 and utilizes funds approved by the State Administration Board.
View the file
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
STATE OF MICHIGAN PROCUREMENT
Department Technology, Management and Budget Central Procurement Services 320 S Walnut Street Lansing, MI 48933 P.O. Box 30026, Lansing, MI 48909
CONTRACT CHANGE NOTICE
CONTRACT SUMMARY
Treasury Performance Testing Services
INITIAL EFFECTIVE DATE INITIAL EXPIRATION DATE INITIAL AVAILABLE OPTIONS EXPIRATION DATE
BEFORE
June 1, 2022 May 31, 2027 5 - 1 Year May 31, 2027
PAYMENT TERMS DELIVERY TIMEFRAME
ALTERNATE PAYMENT OPTIONS EXTENDED PURCHASING
MINIMUM DELIVERY REQUIREMENTS
NA
P-Card Direct Voucher (PRC) Other Yes No
38777 6 Mile Rd
Livonia MI 48152
Caleb Billingsley
770-377-9303 caleb.billingsley@foulkconsulting.com
VS0035898
Foulk Consulting Services Inc
C O
N
TR
A C
TO
R
517-636-5052 plinel@michigan.gov
Emily Massa
(517) 897-7321
DTMB
Lucy Pline DTMB massae@michigan.gov
C ontract
A dm inistrator Program M anagerSTA
TE
NANET 45
Change Notice Number 8
Contract Number MA220000000454 to
DESCRIPTION OF CHANGE NOTICE
OPTION EXTENSIONLENGTH OF OPTION LENGTH OF EXTENSION REVISED EXP. DATE
CURRENT VALUE VALUE OF CHANGE NOTICE ESTIMATED AGGREGATE CONTRACT VALUE
0 Years
$800,886.00 ($50,886.00) $750,000.00
CHANGE NOTICE NO. 8 TO CONTRACT NO. MA220000000454
FOR THE CONTRACTOR:
Foulk Consulting Services Inc Company Name
Authorized Agent Signature
Authorized Agent (Print or Type)
Date
FOR THE STATE:
Signature
DESCRIPTION
Effective September 25th, 2025, the following changes are hereby incorporated into this contract:
1. The value of the contract is decreased by 50,886.00 to correct change notice 7.
2. $15,000 has been added to the scope of work issued in Change Notice 7 for software and scripting time. The modified scope of work is attached and now incorporated into this contract.
All other terms, conditions, specifications and pricing remain the same. Per contractor and agency agreement, and DTMB Central Procurement approval.
DTMB Central Procurement Services Agency
Date
Name and Title
Michigan.gov/MiProcurement
Version 2021-1 1 of 10
STATEMENT OF WORK – CN 8
Project Title:
25.12 Release Performance Test on MTO
Period of Coverage:
July-November 2025
Requesting Department:
DTMB Agency Services Supporting Treasury
Date:
6/18/2025 Change Notice Date:
8/21/2025
Agency Project Manager:
Tamra Phinney
Phone:
517-449-5319
DTMB Project Manager:
Caroline Benveniste SAM 15
Phone:
517-898-6071
Brief description of services to be provided:
BACKGROUND:
Each year Michigan Treasury Online (MTO) is performance tested to ensure that changes made during the year did not degrade performance.
PROJECT OBJECTIVE:
To execute a performance test on MTO during the year-end. MTO performance will be tested for appropriate speed of page render. The projects below have changes that will be tested. All scripts exist and will be reused.
DWT-298 Business YE DWT-303 MTO/MiLogin Reintegration
SCOPE OF WORK:
See Appendix A
Version 2021-1 2 of 10
TASKS:
Technical support from Foulk is required to deliver with the following tasks:
Task Responsible Team
Start Date End Date
Baseline Script Preparation (8 existing MTO scripts)
Foulk 7/17/2025 8/26/2025
Baseline Test Execution in NeoLoad (1 Test - MTO)
Foulk 8/27/2025 8/29/2025
Final Test Script Preparation (8 existing MTO scripts)
Foulk 10/1/2025 11/11/2025
Final Test Execution (Max 3 testing days - MTO)
Foulk 11/12/2025 11/21/2025
DELIVERABLES:
Deliverables will not be considered complete until the Agency Project Manager has formally accepted them. Deliverables for this project include:
1. Eight performance scripts in NeoLoad updated before the Baseline test.
2. Baseline test execution in NeoLoad and Baseline Report within 1 business day of test execution.
3. Eight MTO performance scripts updated after the Baseline test and before the
Final test.
4. Final test execution and reports within 1 business day of test execution (up to
3 days of testing).
5. Final executive summary report after all testing is complete (received within 3 business days after final test).
ACCEPTANCE CRITERIA:
A successful performance test where:
1. Less than a 3% script error rate. (Data or system errors not included.)
2. Availability of a resource to run tests for baseline and 3 days of final testing.
3. Final executive summary report delivery (received within 3 business days after final test).
Version 2021-1 3 of 10
PROJECT CONTROL AND REPORTS:
A bi-weekly progress report must be submitted to the Agency and DTMB Project Managers throughout the life of this project. This report may be submitted with the billing invoice. Each bi-weekly progress report must contain the following:
1. Hours: Indicate the number of hours expended during the past two weeks, and the cumulative total to date for the project. Also state whether the remaining hours are sufficient to complete the project.
2. Accomplishments: Indicate what was worked on and what was completed during the current reporting period.
3. Funds: Indicate the amount of funds expended during the current reporting period, and the cumulative total to date for the project.
PAYMENT SCHEDULE:
NOTE: Payment can be based upon:
• Satisfactory acceptance of each deliverable by DTMB Project Manager.
• All invoices must include the purchase order number.
• Payment will be made on a Satisfactory acceptance of each deliverable basis.
DTMB will pay CONTRACTOR upon receipt of properly completed invoice(s) which shall be submitted to the billing address on the State issued purchase order not more often than monthly. DTMB Accounts Payable area will coordinate obtaining Agency and DTMB Project Manager approvals. All invoices should reflect actual work completed by payment date and must be approved by the Agency and DTMB Project Manager prior to payment. The invoices shall describe and document to the State’s satisfaction a description of the work performed, the progress of the project, and fees. When expenses are invoiced, receipts will need to be provided along with a detailed breakdown of each type of expense.
Payment shall be considered timely if made by DTMB within forty-five (45) days after receipt of properly completed invoices.
Version 2021-1 4 of 10
PRICING:
Task/Milestone Start Date
End Date
Deliverable(s) Milestone Fixed Fee
Cost
MTO Baseline Script Preparation (8 scripts)
7/17/2025 8/15/2025 8 performance scripts in NeoLoad updated before the Baseline test.
$25,355.00
Baseline MTO (1 Test)
8/18/2025 8/20/2025 Baseline test execution and Baseline Report.
$4,246.00
MTO Script Remediation (8 Scripts)
10/1/2025 11/11/2025 8 performance scripts updated after the Baseline test and before the final test.
$11,660.00
Final Test Run (Max 3 testing days)
11/12/2025 11/21/2025 Final test execution (up to 3 days) and Final performance test results report.
$ 9,625.00
Total: $ 50,886.00
Change Notice:
Additional time is required for scripting in a new tool.
Item Cost Software – 30 days for final test $3,800 Additional Scripting time $11,200 Total $15,000
EXPENSES:
The State will NOT pay for any travel expenses, including hotel, mileage, meals, parking, etc.
PROJECT CONTACTS:
Version 2021-1 5 of 10
The designated Agency Project Manager is:
Name Tamra Phinney Department Department Treasury Area Business Taxpayer Services Division Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-449-5319 Email Address phineyt@michigan.gov
The designated DTMB Project Manager is:
Name Caroline Benveniste SAM 15 Department Agency Services supporting Treasury Area Agency Support Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-898-6071 Email Address benvenistec@michigan.gov
AGENCY (DTMB) RESPONSIBILITIES:
Task Responsible Team
Start Date End Date
Point MT1 to EQ1-291 DB DTMB - Basis 7/7/2025 7/14/2025
Open MT1 link to public DTMB - Basis 7/7/2025 7/14/2025
Script Data Prep (after 25.8 client build)
DTMB - Testing 7/15/2025 7/16/2025
Baseline Script Preparation Support DTMB 7/17/2025 8/26/2025
Baseline Test Execution Support DTMB 8/27/2025 8/29/2025
Final Test Script Preparation Support DTMB 10/1/2025 11/11/2025
Full Script Data Prep (After 25.12 Client Build 2)
DTMB - Testing 10/29/2025 10/31/2025
Final Test Execution Support DTMB 11/12/2025 11/21/2025
Close MT1 link to public DTMB - Basis 11/24/2025 11/26/2025
Point MT1 to ET1-400 DTMB - Basis 11/24/2025 11/26/2025
Version 2021-1 6 of 10
LOCATION OF WHERE THE WORK IS TO BE PERFORMED:
Consultants will work at:
Remote
EXPECTED CONTRACTOR WORK HOURS AND CONDITIONS:
Work hours are not to exceed eight (8) hours a day, forty (40) hours a week. Normal working hours of 8:00 am to 5:00 pm are to be observed unless otherwise agreed to in writing.
No overtime will be permitted.
This purchase order is a release from Contract Number MA220000000454. This purchase order, statement of work, and the terms and conditions of Contract Number MA220000000454 constitute the entire agreement between the State and the Contractor.
PROJECT PLAN:
See Appendix A.
APPENDIX A:
Test Schedule and Responsibilities
Task Responsible Team
Start Date End Date
Point MT1 to EQ1-291 DB DTMB - Basis 7/7/2025 7/14/2025
Open MT1 link to public DTMB - Basis 7/7/2025 7/14/2025
Script Data Prep (after 25.8 client build)
DTMB - Testing 7/15/2025 7/16/2025
Baseline Script Preparation (8 existing MTO scripts)
Foulk 7/17/2025 8/15/2015
Baseline Script Preparation Support DTMB 7/17/2025 8/15/2015
Baseline Test Execution in NeoLoad(1 Test - MTO)
Foulk 8/18/2025 8/26/2025
Baseline Test Execution Support DTMB 8/27/2025 8/29/2025
Final Test Script Preparation (8 existing MTO scripts)
Foulk 10/1/2025 11/11/2025
Version 2021-1 7 of 10
Final Test Script Preparation Support DTMB 10/1/2025 11/11/2025
Full Script Data Prep (After 25.12 Client Build 2)
DTMB - Testing 10/29/2025 10/31/2025
Final Test Execution Max 3 testing days - MTO)
Foulk 11/12/2025 11/21/2025
Final Test Execution Support DTMB 11/12/2025 11/21/2025
Known System Downtime (Dates are subject to change):
Reason Start Date End Date
August QA Patching 8/12/2025 8:00 AM 8/12/2025 5:00 PM
August Pre-Prod Patching 8/22/2025 8:00 AM 8/22/2025 5:00 PM
September QA Patch 9/9/2025 8:00 AM 9/9/2025 5:00 PM
25.12 Client Build 1 9/15/2025 8:00 AM 9/17/2025 5:00 PM
September Pre-Prod Patch 9/18/2025 8:00 AM 9/18/2025 5:00 PM
October QA Patching 10/14/2025 8:00 AM 10/14/2025 5:00 PM
October Pre-Prod Patching 10/23/2025 8:00 AM 10/23/2025 5:00 PM
25.12 Client Build 2 10/27/2025 8:00 AM 10/28/2025 5:00 PM
November QA Patching 11/10/2025 8:00 AM 11/10/2025 5:00 PM
Veterans Day (system up no SOM resources)
11/11/2025 11/11/2025
November Pre-Prod Patching 11/18/2025 8:00 AM 11/18/2025 5:00 PM
Test Plan
The performance test will be performed with 1650 concurrent users for a duration of 2.5 hours. The ramp up time is 2 users every 1.5 seconds totaling about 21 min, with the full load sustained for 2 hours and 9 min.
• MTO Users: 1,650
Version 2021-1 8 of 10
*During all test Basis needs to run a script to clean up the W-2 and Bulk files or the server will fill, and the scripts will fail
1. Baseline
• 1 baseline test of the above scripts will be performed before client build of the release.
The test is comprised of the following scripts:
Script Name Percentage of users
Number of
Users
Approx.
Num of submissions
MTO - FTE File Return with Uploaded File (see sizes documented below) 5% 83 1392
MTO - FTE File Return with Manual entry of 100 rows 5% 83 1464
MTO - SUW File Annual Return & View Print Return 28% 462 3996
MTO - SUW file Return – Monthly & Quarterly with Credit Schedule 27% 446 8080
MTO - Bulk e-file upload 5% 83 1176
MTO - View, Edit and Submit ESA file return 12% 198 4073
MTO - W2 Upload (see sizes documented below) – Both EFW2 & SUW covered 13% 215 5411
MTO - ESA Upload (10 Parcel File) 5% 83 346
W2 File Sizes
File Type Percentage of users
Number of Users
Upload 100KB W2 PDF 10 Times 26% 56
Version 2021-1 9 of 10
Upload 3MB W2 zip/rar 15% 32
Upload 10 MB EFW2 txt 15% 32
Upload 50 MB EFW2 txt 15% 32
Upload 100 MB EFW2 txt 20% 43
Upload 500MB EFW2 txt file with zip compression = ~6.0MB zip file 2% 4
Upload 500MB EFW2 txt file with 7z compression = ~235KB 7z file 2% 4
Upload 1GB EFW2 txt file with zip compression = ~12MB zip file 2% 4
Upload 1GB EFW2 txt file with 7z compression = ~335KB 7z file 3% 6
FTE File Sizes of users
Number of Users
1000 5% 4
750 5% 4
500 30% 25
200 60% 50
2. Final Test Run A Maximum of 3 test runs may be required to complete the final test run.
All scripts listed in the baseline will be executed for the Final Test, however the scripts will be updated to use the next tax year, no new scripts are expected.
• If the first test matches the baseline scripts and no concerns are raised no further performance test will be needed
Version 2021-1 10 of 10
Testing Tools
Vendor will use Tricentis Neoload and other load test tools. The State owns the Neoload licenses, and Foulk will use the licenses during testing.
Roles and Responsibility
Individual/Vender Responsibility
Raja Ramaswamy Project Manager
Heather Amos Performance Test SME
Manas Panda MTO Developer Manager
Srinivas Parkala Technical Architect
Charles King Foulk Consulting Account Manager
Sandy Schubert Foulk Consulting Project Manager
Out Of Scope
• Siebel is not part of this performance test
• IVR is not part of this performance test
• No scripts are needed for TP-606
• All eService applications are not part of this performance test
STATE OF MICHIGAN
ENTERPRISE PROCUREMENT
DTMB
320 S Walnut Street Lansing, MI 48933 P.O. Box 30026, Lansing, MI 48909
CONTRACT CHANGE NOTICE
CONTRACT SUMMARY
Treasury Performance Testing Services
INITIAL EFFECTIVE DATE INITIAL EXPIRATION DATE INITIAL AVAILABLE OPTIONS EXPIRATION DATE
BEFORE
June 1, 2022 May 31, 2027 5 - 1 Year May 31, 2027
PAYMENT TERMS DELIVERY TIMEFRAME
ALTERNATE PAYMENT OPTIONS EXTENDED PURCHASING
MINIMUM DELIVERY REQUIREMENTS
NA
P-Card Direct Voucher (PRC) Other Yes No
38777 6 Mile Rd
Livonia MI 48152
Caleb Billingsley
770-377-9303 caleb.billingsley@foulkconsulting.com
VS0035898
Foulk Consulting Services Inc
C O
N
TR
A C
TO
R
517-636-5052 plinel@michigan.gov
Emily Massa
517-897-7321
DTMB
Lucy Pline DTMB massae@michigan.gov
C ontract
A dm inistrator Program M anagerSTA
TE
NANET 45
Change Notice Number 7
Contract Number MA220000000454 to
DESCRIPTION OF CHANGE NOTICE
OPTION EXTENSIONLENGTH OF OPTION LENGTH OF EXTENSION REVISED EXP. DATE
CURRENT VALUE VALUE OF CHANGE NOTICE ESTIMATED AGGREGATE CONTRACT VALUE
$750,000.00 $50,886.00 $800,886.00
DESCRIPTION
Effective August 1st, 2025, $50,8860.0 is added to this contract for the attached scope of work.
All other terms, conditions, specifications and pricing remain the same. Per contractor and agency agreement, and DTMB Central Procurement approval.
Version 2021-1 1 of 10
STATEMENT OF WORK – CN 7
Project Title:
25.12 Release Performance Test on MTO
Period of Coverage:
July-November 2025
Requesting Department:
DTMB Agency Services Supporting Treasury
Date:
6/18/2025
Agency Project Manager:
Tamra Phinney
Phone:
517-449-5319
DTMB Project Manager:
Caroline Benveniste SAM 15
Phone:
517-898-6071
Brief description of services to be provided:
BACKGROUND:
Each year Michigan Treasury Online (MTO) is performance tested to ensure that changes made during the year did not degrade performance.
PROJECT OBJECTIVE:
To execute a performance test on MTO during the year-end. MTO performance will be tested for appropriate speed of page render. The projects below have changes that will be tested. All scripts are existing and will be reused.
DWT-298 Business YE DWT-303 MTO/MiLogin Reintegration
SCOPE OF WORK:
See Appendix A
Version 2021-1 2 of 10
Technical support from Foulk is required to deliver with the following tasks:
Task Responsible Team
Start Date End Date
Baseline Script Preparation (8 existing MTO scripts)
Foulk 7/17/2025 8/15/2015
Baseline Test Execution in NeoLoad (1 Test - MTO)
Foulk 8/18/2025 8/20/2025
Final Test Script Preparation (8 existing MTO scripts)
Foulk 10/1/2025 11/11/2025
Final Test Execution in NeoLoad (Max 3 testing days - MTO)
Foulk 11/12/2025 11/21/2025
DELIVERABLES:
Deliverables will not be considered complete until the Agency Project Manager has formally accepted them. Deliverables for this project include:
1. Eight performance scripts in NeoLoad updated before the Baseline test.
2. Baseline test execution in NeoLoad and Baseline Report within 1 business day of test execution.
3. Eight MTO performance scripts in NeoLoad updated after the Baseline test and before the Final test.
4. Final test execution in NeoLoad and reports within 1 business day of test execution (up to 3 days of testing).
5. Final executive summary report after all testing is complete (received within 3 business days after final test).
ACCEPTANCE CRITERIA:
A successful performance test where:
1. Less than a 3% script error rate. (Data or system errors not included.)
2. Availability of a resource to run tests for baseline and 3 days of final testing.
3. Final executive summary report delivery (received within 3 business days after final test).
Version 2021-1 3 of 10
PROJECT CONTROL AND REPORTS:
A bi-weekly progress report must be submitted to the Agency and DTMB Project Managers throughout the life of this project. This report may be submitted with the billing invoice. Each bi-weekly progress report must contain the following:
1. Hours: Indicate the number of hours expended during the past two weeks, and the cumulative total to date for the project. Also state whether the remaining hours are sufficient to complete the project.
2. Accomplishments: Indicate what was worked on and what was completed during the current reporting period.
3. Funds: Indicate the amount of funds expended during the current reporting period, and the cumulative total to date for the project.
PAYMENT SCHEDULE:
NOTE: Payment can be based upon:
• Satisfactory acceptance of each deliverable by DTMB Project Manager.
• All invoices must include the purchase order number.
• Payment will be made on a Satisfactory acceptance of each deliverable basis.
DTMB will pay CONTRACTOR upon receipt of properly completed invoice(s) which shall be submitted to the billing address on the State issued purchase order not more often than monthly. DTMB Accounts Payable area will coordinate obtaining Agency and DTMB Project Manager approvals. All invoices should reflect actual work completed by payment date and must be approved by the Agency and DTMB Project Manager prior to payment. The invoices shall describe and document to the State’s satisfaction a description of the work performed, the progress of the project, and fees. When expenses are invoiced, receipts will need to be provided along with a detailed breakdown of each type of expense.
Payment shall be considered timely if made by DTMB within forty-five (45) days after receipt of properly completed invoices.
PRICING:
Task/Milestone Start Date
End Date
Deliverable(s) Milestone Fixed Fee
Cost
MTO Baseline Script Preparation (8 scripts)
7/17/2025 8/15/2025 8 performance scripts in NeoLoad updated before the Baseline test.
$25,355.00
Version 2021-1 4 of 10
Baseline MTO (1 Test)
8/18/2025 8/20/2025 Baseline test execution and Baseline Report.
$4,246.00
MTO Script Remediation (8 Scripts)
10/1/2025 11/11/2025 8 performance scripts in NeoLoad updated after the Baseline test and before the final test.
$11,660.00
Final Test Run (Max 3 testing days)
11/12/2025 11/21/2025 Final test execution (up to 3 days) and Final performance test results report.
$ 9,625.00
Total: $ 50,886.00 parking, etc.
PROJECT CONTACTS:
The designated Agency Project Manager is:
Name Tamra Phinney Department Department Treasury Area Business Taxpayer Services Division Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-449-5319 Email Address phineyt@michigan.gov
The designated DTMB Project Manager is:
Name Caroline Benveniste SAM 15 Department Agency Services supporting Treasury Area Agency Support Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-898-6071 Email Address benvenistec@michigan.gov
Version 2021-1 5 of 10
AGENCY (DTMB) RESPONSIBILITIES:
Start Date End Date
Point MT1 to EQ1-291 DB DTMB - Basis 7/7/2025 7/14/2025
Open MT1 link to public DTMB - Basis 7/7/2025 7/14/2025
Script Data Prep (after 25.8 client build)
DTMB - Testing 7/15/2025 7/16/2025
Baseline Script Preparation Support DTMB 7/17/2025 8/15/2015
Baseline Test Execution Support DTMB 8/18/2025 8/20/2025
Final Test Script Preparation Support DTMB 10/1/2025 11/11/2025
Full Script Data Prep (After 25.12 Client Build 2)
DTMB - Testing 10/29/2025 10/31/2025
Final Test Execution Support DTMB 11/12/2025 11/21/2025
Remote
EXPECTED CONTRACTOR WORK HOURS AND CONDITIONS:
Work hours are not to exceed eight (8) hours a day, forty (40) hours a week. Normal working hours of 8:00 am to 5:00 pm are to be observed unless otherwise agreed to in writing.
No overtime will be permitted.
This purchase order is a release from Contract Number MA220000000454. This purchase order, statement of work, and the terms and conditions of Contract Number MA220000000454 constitute the entire agreement between the State and the Contractor.
Version 2021-1 6 of 10
PROJECT PLAN:
See Appendix A.
APPENDIX A:
Test Schedule and Responsibilities
Task Responsible Team
Start Date End Date
Point MT1 to EQ1-291 DB DTMB - Basis 7/7/2025 7/14/2025
Open MT1 link to public DTMB - Basis 7/7/2025 7/14/2025
Script Data Prep (after 25.8 client build)
DTMB - Testing 7/15/2025 7/16/2025
Baseline Script Preparation (8 existing MTO scripts)
Foulk 7/17/2025 8/15/2015
Baseline Script Preparation Support DTMB 7/17/2025 8/15/2015
Baseline Test Execution in NeoLoad(1 Test - MTO)
Foulk 8/18/2025 8/20/2025
Baseline Test Execution Support DTMB 8/18/2025 8/20/2025
Final Test Script Preparation (8 existing MTO scripts)
Foulk 10/1/2025 11/11/2025
Final Test Script Preparation Support DTMB 10/1/2025 11/11/2025
Full Script Data Prep (After 25.12 Client Build 2)
DTMB - Testing 10/29/2025 10/31/2025
Final Test Execution in NeoLoad (Max 3 testing days - MTO)
Foulk 11/12/2025 11/21/2025
Final Test Execution Support DTMB 11/12/2025 11/21/2025
Version 2021-1 7 of 10
Known System Downtime (Dates are subject to change):
Reason Start Date End Date
August QA Patching 8/12/2025 8:00 AM 8/12/2025 5:00 PM
August Pre-Prod Patching 8/21/2025 8:00 AM 8/21/2025 5:00 PM
September QA Patch 9/9/2025 8:00 AM 9/9/2025 5:00 PM
25.12 Client Build 1 9/15/2025 8:00 AM 9/17/2025 5:00 PM
September Pre-Prod Patch 9/18/2025 8:00 AM 9/18/2025 5:00 PM
October QA Patching 10/14/2025 8:00 AM 10/14/2025 5:00 PM
October Pre-Prod Patching 10/23/2025 8:00 AM 10/23/2025 5:00 PM
25.12 Client Build 2 10/27/2025 8:00 AM 10/28/2025 5:00 PM
November QA Patching 11/10/2025 8:00 AM 11/10/2025 5:00 PM
Veterans Day (system up no SOM resources)
11/11/2025 11/11/2025
November Pre-Prod Patching 11/18/2025 8:00 AM 11/18/2025 5:00 PM
The performance test will be performed with 1650 concurrent users for a duration of 2.5 hours. The ramp up time is 2 users every 1.5 seconds totaling about 21 min, with the full load sustained for 2 hours and 9 min.
• MTO Users: 1,650 *During all test Basis needs to run a script to clean up the W-2 and Bulk files or the server will fill, and the scripts will fail
1. Baseline
• 1 baseline test of the above scripts will be performed before client build of the
Version 2021-1 8 of 10
The test is comprised of the following scripts:
Script Name Percentage of users
Number of
Users
Approx.
Num of submissions
MTO - FTE File Return with Uploaded File (see sizes documented below) 5% 83 1392
MTO - FTE File Return with Manual entry of 100 rows 5% 83 1464
MTO - SUW File Annual Return & View Print Return 28% 462 3996
MTO - SUW file Return – Monthly & Quarterly with Credit Schedule 27% 446 8080
MTO - Bulk e-file upload 5% 83 1176
MTO - View, Edit and Submit ESA file return 12% 198 4073
MTO - W2 Upload (see sizes documented below) – Both EFW2 & SUW covered 13% 215 5411
MTO - ESA Upload (10 Parcel File) 5% 83 346 of users
Number of Users
Upload 100KB W2 PDF 10 Times 26% 56
Upload 3MB W2 zip/rar 15% 32
Upload 10 MB EFW2 txt 15% 32
Upload 50 MB EFW2 txt 15% 32
Upload 100 MB EFW2 txt 20% 43
Upload 500MB EFW2 txt file with zip compression = ~6.0MB zip file 2% 4
Upload 500MB EFW2 txt file with 7z compression = ~235KB 7z file 2% 4
Version 2021-1 9 of 10
Upload 1GB EFW2 txt file with zip compression = ~12MB zip file 2% 4 of users
Number of Users
1000 5% 4
750 5% 4
All scripts listed in the baseline will be executed for the Final Test, however the scripts will be updated to use the next tax year, no new scripts are expected.
Testing Tools
Vendor will use Tricentis Neoload load test tools. The State owns the licenses, and Foulk will use the licenses during testing.
Raja Ramaswamy Project Manager
Heather Amos Performance Test SME
Manas Panda MTO Developer Manager
Version 2021-1 10 of 10
Srinivas Parkala Technical Architect
Charles King Foulk Consulting Account Manager
Sandy Schubert Foulk Consulting Project
• Siebel is not part of this performance test
• IVR is not part of this performance test
• No scripts are needed for TP-606
• All eService applications are not part of this performance test
Department of Technology, Management, and Budget
525 W. ALLEGAN ST., LANSING, MICHIGAN 48913
P.O. BOX 30026 LANSING, MICHIGAN 48909
CONTRACT CHANGE NOTICE
CONTRACT SUMMARY
Treasury Performance Testing Services
INITIAL EFFECTIVE DATE INITIAL EXPIRATION DATE INITIAL AVAILABLE OPTIONS EXPIRATION DATE
BEFORE
June 1, 2022 May 31, 2027 5 - 12 Months May 31, 2027
PAYMENT TERMS DELIVERY TIMEFRAME
ALTERNATE PAYMENT OPTIONS EXTENDED PURCHASING
MINIMUM DELIVERY REQUIREMENTS
NA
P-Card Direct Voucher (PRC) Other Yes No
38777 6 Mile Rd
Livonia MI 48152
Caleb Billingsley
770-377-9303 caleb.billingsley@foulkconsulting.com
VS0035898
Foulk Consulting Services Inc
C O
N
TR
A C
TO
R
517-636-5052 plinel@michigan.gov
Melissa Beck
(517) 897-1502
DTMB
Lucy Pline DTMB beckm3@michigan.gov
C ontract
A dm inistrator Program M anagerSTA
TE
NANET 45
Change Notice Number 6
Contract Number MA220000000454 to
DESCRIPTION OF CHANGE NOTICE
OPTION EXTENSIONLENGTH OF OPTION LENGTH OF EXTENSION REVISED EXP. DATE
CURRENT VALUE VALUE OF CHANGE NOTICE ESTIMATED AGGREGATE CONTRACT VALUE
$717,120.00 $0.00 $717,120.00
DESCRIPTION
Effective July 2, 2024, this contract has been amended as follows:
The attached SOW is hereby incorporated into this Contract to complete the DWT-273 Business YE and
ECC Upgrade Performance Test. No additional funds are needed at this time.
All other terms, conditions, specifications, and pricing remain the same. Per contractor and agency agreement, and DTMB Central Procurement Services approval.
Version 2021-1 1 of 12
STATEMENT OF WORK -
IT CHANGE NOTICE
Project Title:
DWT-273 Business YE and ECC Upgrade Performance Test
Period of Coverage:
July-November 2024
Requesting Department:
Agency Services Supporting Treasury
Date:
6/12/2024
Agency Project Manager:
Tamra Phinney SAM 15
Phone:
517-449-5319
DTMB Project Manager:
Caroline Benveniste SAM 15
Phone:
517-898-6071
Brief description of services to be provided:
BACKGROUND:
Each year Michigan Treasury Online (MTO) is performance tested to ensure that changes made during the year did not degrade performance.
PROJECT OBJECTIVE:
To execute a performance test on MTO with year-end project and the ECC upgrade.
MTO performance will be patched as indicated from the performance test.
SCOPE OF WORK:
See Appendix A
Technical support from Foulk is required to assist with the following tasks:
Task Start Date End Date
Baseline Script Preparation (8 existing scripts)
7/15/2024 8/2/2024
Baseline (1 Test) 8/5/2024 8/12/2024
Final Test: 0 New Script creation /8 Existing Script Preparation
9/25/2024 11/15/2024
Final Test Run (Max 3 testing days) 11/18/2024 11/22/2024
Version 2021-1 2 of 12
DELIVERABLES:
Deliverables will not be considered complete until the Agency Project Manager has formally accepted them. Deliverables for this project include:
Updated performance scripts in NeoLoad.
Performance test results report.
ACCEPTANCE CRITERIA:
A successful performance test where:
1. Less than a 3% script error rate. (Data or system errors not included.)
2. Availability of a resource to run tests for baseline and 3 days of final testing.
PROJECT CONTROL AND REPORTS:
A bi-weekly progress report must be submitted to the Agency and DTMB Project Managers throughout the life of this project. This report may be submitted with the billing invoice. Each bi-weekly progress report must contain the following:
1. Hours: Indicate the number of hours expended during the past two weeks, and the cumulative total to date for the project. Also state whether the remaining hours are sufficient to complete the project.
2. Accomplishments: Indicate what was worked on and what was completed during the current reporting period.
3. Funds: Indicate the amount of funds expended during the current reporting period, and the cumulative total to date for the project.
SPECIFIC DEPARTMENT STANDARDS:
Agency standards, if any, in addition to DTMB standards.
PAYMENT SCHEDULE:
NOTE: Payment can be based upon:
• Satisfactory acceptance of each deliverable
• All invoices must include the purchase order number.
• Payment will be made on a Satisfactory acceptance of each deliverable basis.
DTMB will pay CONTRACTOR upon receipt of properly completed invoice(s) which shall be submitted to the billing address on the State issued purchase order not more often than monthly. DTMB Accounts Payable area will coordinate obtaining Agency and DTMB Project Manager approvals. All invoices should reflect actual work completed by payment date and must be approved by the Agency and DTMB Project Manager prior to payment. The invoices shall describe and document to the State’s satisfaction a description of the work performed, the
Version 2021-1 3 of 12 progress of the project, and fees. When expenses are invoiced, receipts will need to be provided along with a detailed breakdown of each type of expense.
Payment shall be considered timely if made by DTMB within forty-five (45) days after receipt of properly completed invoices.
PRICING
Task/Milestone Start Date End Date Deliverable(s) Milestone Fixed Fee Cost
Baseline Script Preparation (8 existing scripts)
7/15/2024 8/2/2024 8 Scripts Prepared for Testing
$22,330.00
Baseline (1 Test) 8/5/2024 8/12/2024 Baseline Performance Test and Baseline Report
$ 9,130.00
Existing Script Preparation for 8 scripts (0 new scripts)
9/25/2024 11/15/2024 8 Scripts Prepared for Testing
$ 9,625.00
Final Test Run (Max 3 testing days)
11/18/2024 11/22/2024 Final Upgrade Performance Test and Final Report
$ 8,882.50
Total: $49,967.50 parking, etc.
PROJECT CONTACTS:
The designated Agency Project Manager is:
Name Tamra Phinney SAM 15 Department Department of Treasury Area Business Taxpayer Services Division Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-449-5319 Email Address phinneyt@michigan.gov
Version 2021-1 4 of 12
The designated DTMB Project Manager is:
Name Caroline Benveniste Department Agency Services supporting Treasury Area Agency Support Building/Floor Operations Center Address 7285 Parsons Ave City/State/Zip Lansing MI Phone Number 517-898-6071 Email Address benvenistec@michigan.gov
AGENCY (DTMB) RESPONSIBILITIES:
Task Start Date End Date
Script Data Prep 7/9/2024 7/12/2024
Assist in Baseline Script Preparation as needed (8 existing scripts)
7/15/2024 8/2/2024
Monitor Baseline (1 Test) 8/5/2024 8/12/2024
Assist in Final Test Existing Scripts Preparation (8 scripts)
9/25/2024 11/15/2024
Full Script Data Prep (After 24.12 Client Build 2)
10/30/2024 11/6/2024
Monitor/Troubleshoot during Final Test Run (Max 3 testing days)
11/18/2024 11/22/2024
Remote
EXPECTED CONTRACTOR WORK HOURS AND CONDITIONS:
Work hours are not to exceed eight (8) hours a day, forty (40) hours a week. Normal working hours of 8:00 am to 5:00 pm are to be observed unless otherwise agreed to in writing.
No overtime will be permitted.
This purchase order is a release from Contract Number MA220000000454. This purchase order, statement of work, and the terms and conditions of Contract Number MA220000000454 constitute the entire agreement between the State and the Contractor.
Version 2021-1 5 of 12
PROJECT PLAN:
See Appendix A.
APPENDIX A:
State of Michigan
24.12.1 Year End
Performance Test Strategy
GENERAL INFORMATION
System or Project ID/Acronym: • DWT_237 Creation Date: 6/3/2024
Client Agency: Treasury Modification Date:
Author(s): Heather Amos DTMB Authorized by: Caroline Benveniste
PRIVACY INFORMATION
This document may contain information of a sensitive nature. This information should not be given to persons other than those who are involved with this system/project or who will become involved during its lifecycle.
REVISION HISTORY
The Project Manager will maintain this information and provide updates as required. All updates to the Project Management Plan and component plans should be documented in this section.
Revision Date Author Section(s) Summary
OVERVIEW
The purpose of the Performance Test Strategy is to communicate the performance testing approach to all appropriate stakeholders (project team, business owners, sponsors and management). This document identifies the types of tests that will be
Version 2021-1 6 of 12 performed, tools are needed to execute the tests, when each of those tests are performed, who is responsible for those tests and how the results of those tests are to be recorded and communicated. By identifying and defining these testing assets earlier in the process, it should assist with budget and resource estimations/allocations. This document may pertain to a single project, a program or an organization.
Definitions for test types and test roles can be found in Appendix A.
TESTING STRATEGY
Test Environments
• The Performance test will be executed in the public facing MTO MQ1/EQ1-291 Test environment.
• Load Generation will come from outside the SOM network from TriCentus/NeoLoad cloud generation machines.
• Scripting and the controller will be done from Foulk machines.
Test Data Management
Dummy Test data will be created by the MTO and Batch team prior to the performance test.
The maximum concurrent user load in production is 1500 users, test will be done with 10% increase, 1650 users, to account for any additional usage in the next year.
Version 2021-1 7 of 12
Test Schedule and Responsibilities
Team
Start Date End Date
Script Data Prep (after 24.8 client build)
DTMB 7/9/2024 7/12/2024
Baseline Script Preparation (8 existing scripts)
DTMB/Foulk 7/15/2024 8/2/2024
Baseline (1 Test) DTMB/Foulk 8/5/2024 8/12/2024
Final Test: 0 New Script creation /8 Existing Script Preparation *Will only have partial data till full script data prep is completed
DTMB/Foulk 9/25/2024 11/15/2024
Full Script Data Prep (After 24.12 Client Build 2)
DTMB 10/30/2024 11/6/2024
Final Test Run (Max 3 testing days) DTMB/Foulk 11/18/2024 11/22/2024
Version 2021-1 8 of 12
Known System Downtime (Dates are subject to change):
Reason Start Date End Date
July QA Patching 7/9/2024 8AM 7/9/2024 7PM
August QA Patching 8/13/2024 8AM 8/13/2024 7PM
ECC Upgrade 9/2/2024 8AM 9/15/2024 5PM
September QA Patch 9/10/2024 8AM 9/10/2024 7PM
24.12 Client Build 1 9/16/2024 8AM 9/18/2014 11PM
October QA Patching 10/15/2024 8AM 10/15/2024 7PM
24.12 Client Build 2 10/28/2024 8AM 10/29/2024 5PM
November QA Patching 11/12/2024 8AM 11/12/2024 7PM
The performance test will be performed with 1,650 concurrent users for a duration of 2.5 hours only using MTO. The ramp up time is 2 users every 1.5 seconds totaling about 21 min, with the full load sustained for 2 hours and 9 min.
*During all test Basis needs to run a script to clean up the W-2 and Bulk files or the server will fill, and the scripts will fail
1. Baseline
• 1 baseline of the above scripts will be performed before client build of the
The test is comprised of the following scripts:
Script Name Percentage of users
Number of
Users
Approx.
Num of submissions
FTE File Return with Uploaded File (see sizes documented below) 5% 83 1392
Version 2021-1 9 of 12
FTE File Return with Manual entry of 100 rows 5% 83 1464
SUW File Annual Return & View Print Return 28% 462 3996
SUW file Return – Monthly & Quarterly with Credit Schedule 27% 446 8080
Bulk e-file upload 5% 83 1176
View, Edit and Submit ESA file return 12% 198 4073
W2 Upload (see sizes documented below) 13% 215 5411
ESA Upload (10 Parcel File) 5% 83 346 of users
Number of Users
Upload 100KB W2 PDF 10 Times 26% 56
Upload 3MB W2 zip/rar 15% 32
Upload 10 MB EFW2 txt 15% 32
Upload 50 MB EFW2 txt 15% 32
Upload 100 MB EFW2 txt 20% 43
Upload 500MB EFW2 txt file with zip compression = ~6.0MB zip file 2% 4
Upload 500MB EFW2 txt file with 7z compression = ~235KB 7z file 2% 4
Upload 1GB EFW2 txt file with zip compression = ~12MB zip file 2% 4
Version 2021-1 10 of 12
File Type Percentage of users
Number of Users
1000 5% 4
750 5% 4
The same test will be executed for the Final Test that was used for baseline; however the scripts will be updated to use the next tax year.
Testing Tools
Vendor will use proprietary load test tools.
Raja Ramaswamy Project Manager
Heather Amos Performance Test SME
Manas Panda MTO developers
Srinivas Parkala Technical Architect
Charles King Foulk Consulting Project
Version 2021-1 11 of 12
Metrics & Reports
Performance testing reporting will be provided by the vendor(s) within 24 hours of every successfully executed test to determine if SLAs are being met. These reports will also be compared with previous load test if applicable to determine to if the test is similar.
Metric What Phase / Product
Increment will be collected in:
Reporting Frequency
Source of Truth
Audience
Load Test results of server responses times/ Errors and through put
Performance Testing
After every Successfully executed performance test
Performance testing tool(s)
Performance Test SME and system architects
Server Metrics from
DYNATRACE
Performance Testing
After every Successfully executed performance test
DYNATRACE Performance Test SME and system architects
Executive Summary
Performance Testing
End of Performance Testing
Summary of Load test results
Business Owners and Stakeholders
Critical Success factors
Describe the activities or elements that are necessary for the test phases to be successful.
• Proper Staffing of qualified resources to monitor and investigate performance.
• Availability of a production emerging hardware and pre-prod environment
• Proper scheduling for test planning and execution activities
• Dynatrace is installed on servers.
Version 2021-1 12 of 12
• eServices is not part of this performance test.
RISKS AND ISSUES
Risk for Performance Test
Risk
Description Mitigation
Issues for Performance Test
Issue
Description Action Responsible Resource
Due Date
Department of Technology, Management, and Budget
525 W. ALLEGAN ST., LANSING, MICHIGAN 48913
P.O. BOX 30026 LANSING, MICHIGAN 48909
CONTRACT CHANGE NOTICE
CONTRACT SUMMARY
Treasury Performance Testing Services
INITIAL EFFECTIVE DATE INITIAL EXPIRATION DATE INITIAL AVAILABLE OPTIONS EXPIRATION DATE
BEFORE
June 1, 2022 May 31, 2027 5 - 12 Months May 31, 2027
PAYMENT TERMS DELIVERY TIMEFRAME
ALTERNATE PAYMENT OPTIONS EXTENDED PURCHASING
MINIMUM DELIVERY REQUIREMENTS
NA
P-Card Direct Voucher (PRC) Other Yes No
38777 6 Mile Rd
Livonia MI 48152
Caleb Billingsley
770-377-9303 caleb.billingsley@foulkconsulting.com
VS0035898
Foulk Consulting Services, Inc
C O
N
TR
A C
TO
R
517-636-5052 plinel@michigan.gov
Melissa Beck
(517) 897-1502
DTMB
Lucy Pline DTMB beckm3@michigan.gov
C ontract
A dm inistrator Program M anagerSTA
TE
NANET 45
Change Notice Number 5
Contract Number MA220000000454 to
DESCRIPTION OF CHANGE NOTICE
OPTION EXTENSIONLENGTH OF OPTION LENGTH OF EXTENSION REVISED EXP. DATE
CURRENT VALUE VALUE OF CHANGE NOTICE ESTIMATED AGGREGATE CONTRACT VALUE
$717,120.00 $0.00 $717,120.00
DESCRIPTION
Effective 6/21/2024, this contract has been amended as follows:
Caleb Billingsley, Sr. Director, Testing & Automation Practice, is the new contact.
caleb.billingsley@foulkconsulting.com, 770-377-9309.
Category Analyst is changed to Melissa Beck, Beckm3@michigan.gov, 517-897-1502.
All other terms, conditions, specifications, and pricing remain the same. Per contractor and agency agreement, and DTMB Central Procurement Services approval.
C O
N
TR
A C
TO
R
Foulk Consulting Services, Inc
38777 6 Mile Rd
Livonia, MI 48152
Fabiola Conti
734-773-1554 fabiola.conti@foulkconsulting.com
VS0035898
STATE
P ro g ram
M an ager
Lucy Pline TREA
517-636-5052 plinel@michigan.gov C on tract A d m in istrato r Patrick Russell DTMB
(517) 648-7767 russellp2@michigan.gov
STATE OF MICHIGAN
CENTRAL PROCUREMENT SERVICES
Department of Technology, Management, and Budget
320 S. WALNUT ST., LANSING, MICHIGAN 4893
P.O. BOX 30026 LANSING, MICHIGAN 48909
CONTRACT CHANGE NOTICE
Change Notice Number 4 to Contract Number 220000000454
CONTRACT SUMMARY
TREASURY PERFORMANCE TESTING SERVICES
INITIAL EFFECTIVE DATE INITIAL EXPIRATION DATE INITIAL AVAILABLE OPTIONS EXPIRATION DATE
BEFORE
June 1, 2022 May 31, 2027 5 - 1 Year May 31, 2027
PAYMENT TERMS DELIVERY TIMEFRAME
ALTERNATE PAYMENT OPTIONS EXTENDED PURCHASING
☐ P-Card ☐ PRC ☐ Other ☐ Yes ☒ No
MINIMUM DELIVERY REQUIREMENTS
DESCRIPTION OF CHANGE NOTICE
OPTION LENGTH OF OPTION EXTENSION LENGTH OF EXTENSION REVISED EXP. DATE
☐ ☐ May 31, 2027
CURRENT VALUE VALUE OF CHANGE NOTICE ESTIMATED AGGREGATE CONTRACT VALUE
$750,000.00 $0.00 $750,000.00
DESCRIPTION
Effective 12/5/2023, this contract has been amended as follows:
The attached SOW is hereby incorporated into this Contract for the addition of 128 hours needed to complete the remaining work for the year end performance testing. Total cost of this project is $23,680.00. No additional funding is needed at this time;
existing funds are adequate to support this change.
All other terms, conditions, specifications, and pricing remain the same. Per contractor and agency agreement, and DTMB Central Procurement Services approval.
mailto:fabiola.conti@foulkconsulting.com mailto:plinel@michigan.gov mailto:russellp2@michigan.gov RussellP2 Line
RussellP2 Line
CHANGE NOTICE NO. 4 TO CONTRACT NO. 220000000454
Program Managers for
Multi-Agency and Statewide Contracts
AGENCY NAME PHONE EMAIL
DTMB Lucy Pline 517-636-5052 plinel@michigan.gov mailto:plinel@michigan.gov
Version 2021-1 1 of 9
STATEMENT OF WORK -
IT CHANGE NOTICE
Project Title:
Siebel/Hybris Upgrade and YE Project Performance Test
Period of Coverage:
Requesting Department:
Agency Services
Date:
11/20/2023
Agency Project Manager:
Mary Brennan
Phone:
517-256-7910
DTMB Project Manager:
Sherry Yagiela
Phone:
517-636-0130
11/21/2023 UPDATES HIGHLIGHTED
Introduction and Contacts This Statement of Work (SOW) is made as of the date of later signature hereto 09/08/2023 by and between Foulk Consulting Services, Inc. (FCS) and The State of Michigan (Client) and identifies the consulting services FCS will perform for Client.
Client must sign this SOW before FCS begins services delivery. This SOW is an SOW referred to in the Contract 220000000454 between the parties dated June 06, 2022 (the “Contract”).
Client Work Site Address Work to be performed remotely.
Client Name Contacts Automation Testing and Mainframe Delivery Manager DTMB-AS Supporting Treasury Caroline Benveniste C:517-898-6071 BenvenisteC@Michigan.gov FCS Contacts Director of Sales Charles King C:813.585.4660 Charles.King@foulkconsulting.com Director of Performance Engineering Brian Brumfield C: C:937.707.0290 Brian.Brumfield@foulkconsulting.com mailto:BenvenisteC@Michigan.gov mailto:Charles.King@foulkconsulting.com mailto:Brian.Brumfield@foulkconsulting.com
Version 2021-1 2 of 9
President Ryan Foulk C:734.478.1165 Ryan.Foulk@FoulkConsulting.com Director of Project Operations Sandy Schubert C:412.401.4986 Sandy.Schubert@FoulkConsulting.com
Estimated Kickoff Date September 8, 2023
Project Overview Foulk Consulting will construct and execute a large and complex year-end performance test for State of Michigan Treasury Department, integrating testing assets for four different production-ready applications; eServices, Siebel, MTO and Siebel's ability to handle direct traffic from the IVR system. Test creation will be composed of elements from previous years' testing activities, combined with new assets, and testing from inside the State of Michigan firewall as well as from the cloud for customer-facing interfaces.
Scope A very complex, unified performance testing project that combines testing for four State of Michigan applications:
1. eServices
2. MTO
3. IVR
4. Siebel The Performance test will be performed with 3,954 concurrent users putting load on all 4 Treasury applications:
MTO, eServices, IVR via WSDL, and Siebel.
Peak production loads plus an additional 25% will be used for each application under test, with a duration of 3 hours. The ramp-up time is 2 users every 1.5 secondstotaling approximately 37minutes, with full load sustained for approximately 2 hours and 23 minutes.
Testing will take place from a NeoLoad Controller that is inside the State of Michigan firewall, using a combination of Load Generator systems that are both inside the State of Michigan's firewall and also in the cloud, utilizing Tricentis NeoLoad SaaS Load Generator resources.
mailto:Ryan.Foulk@FoulkConsulting.com mailto:Sandy.Schubert@FoulkConsulting.com
Version 2021-1 3 of 9
Applications Under Test eServices:
The following, existing NeoLoad testing scripts must be validated and maintained where necessary, comprised of the following, existing NeoLoad test scripts:
1. Guest- Where’s my Refund& Refund Inquiry
2. Account - Where’s my Refund & Refund Inquiry
3. View Estimated Payments – Guest
4. View Estimated Payments – Account
5. Change My Address-Guest
6. UPE Calculator
7. Link New Account (Will already have MiLogin account)
8. Tax Professional – Submit and Review Inquiry - IIT
9. Tax Professional – Submit and Review Inquiry – Business taxes
10. Tax Professional – Submit and Review Inquiry –General
The following new eServices script will be created:
1. Business Taxes – Account Login – Respond to a letter (SUW) with a file upload submission to get a confirmation number & View Response
Siebel IVR Interface:
The WSDL interface into Siebel will be used to simulate incoming IVR sessions.
An old script may be used if the WSDL interface has remained unchanged since 2020, otherwise re-writing the test script may be required, and any new scripts must be created.
1. Main Business IVR: Accounts With SUW Data
2. Main IIT IVR: Accounts with 2020 returns, in pending Review Siebel:
The following, existing NeoLoad testing scripts must be validated and maintained where necessary, comprised of the following, existing NeoLoad test scripts:
1. Incoming call Details Displayed – Known user details
2. Incoming call Details Displayed – unknown user details, search screen
Version 2021-1 4 of 9
3. Create& Update service request
4. View taxpayer information – tax returns
5. View taxpayer information – refunds
6. View taxpayer information – Web link from SAP for Bus
7. View correspondence in FileNet
8. Create& Update Contact logs
9. Service Request Query
10. Contact Logs Query
MTO:
1. FTE File Return with Uploaded File
2. FTE File Return with Manual entry of 100 rows
3. SUW File Annual Return& View Print Return
4. SUW file Return – Monthly& Quarterly with Credit Schedule
5. Bulk e-file upload
6. View, Edit and Submit ESA file return
7. W2 Upload
8. ESA Upload *MTO file sizes are an element of test data and test design and will be handled as per the SOM test plan
Deliverables
• Scripting as described in the scope
• One report that compares the system performance to a baseline provided by State of Michigan.
• Reports on Load Tests
• NeoLoad Test Project with testing assets at the conclusion of testing, if desired.
• ALL firewall rules that are required for each required connection (including between the servers). This is to include source, destination, and port(s).
• Any deliverable presented as completed will be assumed to be accepted if not refused within a five (5) business day acceptance window.
Project Timelines and Assumptions The estimated duration of this Project is 7.5 weeks.
Version 2021-1 5 of 9
The proposed schedule below assumes a start date of September 8, 2023
Task Responsible Team Start Date End Date MTO data prep of 10 accounts
DTMB 9/8/2023 09/12/2023
MTO script review Foulk 9/13/2023 10/17/2023 New script creation and script review in eServices, Siebel and IVR
DTMB/Foulk 10/01/2023 (But will not have full data until 10/30/2023)
11/10/2023
Script data prep (After 23.12.1 Client
Build 2)
DTMB 10/20/2023 10/27/2023
Final test run (up to 4 test days)
DTMB/Foulk 11/13/2023 11/22/2023
• FCS will provide remote assistance during the term of the engagement.
• FCS’s standard workweek is five days x 8 hours per day (M-F) for remote work.
Unless otherwise agreed, this will be the standard workweek for the duration of the engagement.
• Some environment changes (page assets; javascript, CSS, other files) may require updates to existing scripts
• Account password updates are required, and scripts and data must be updated accordingly
• Testing data will require validation and updates to ensure low failure rates due to data-related issues
• State of Michigan provides access to the system to be scripted
• State of Michigan provides all data required for Foulk to script the documented business processes
• State of Michigan provides unfettered access to a stable (up and un-changing) application instance the same as the system under test for scripting with availability exceptions being communicated to Foulk by State of Michigan
• Any application outages could impact the testing schedule dates
• Foulk will test and validate for functionality and scripting-error-free state of scripts based on workflow documentation provided by the State of Michigan. However, Foulk
Version 2021-1 6 of 9 cannot account for load-related errors, data-related errors, or business process variations that were not included in the scripting requirements.
Script calibrations and data validation Assumptions:
• Foulk to assist with the scoping of data requirements for the testing needs as documented
• Foulk to assist with the validation of the data as related to the scripted business processes as documented
• State of Michigan to create and manage all data required for testing
• State of Michigan to reset any databases or data following a test
• State of Michigan to provide test data so no exposure to production data occurs
Project Resource Description FCS will be providing a consultant filling the roles listed below for remote services delivery.
This is the start of the file's text. The full file is on GovTribe.
File details come from the government source that posted it. Updated .