Attachment C - FCC Auction Bidding System and Support Services - Evaluation Criteria.pdf

Attachment C

EVALUATION CRITERIA AND BASIS FOR AWARD

FCC Auction Bidding System and Support Services

OBJECTIVE

The overall objective of the evaluation is for the Government to ascertain the extent to which it can receive from the Offeror timely, high quality, consistently reliable and successful work as set forth in the Statement of Work. Therefore, Offerors should ensure that their initial proposal constitutes their best proposal in terms of both price and the technical solution being proposed

EVALUATION CRITERIA

Any proposal/quote that fails to provide all information required may not be further considered, especially if the Contacting Officer (CO) determines substantial revisions to the Offeror’s proposal/quote would be required to permit evaluation, or if the incomplete proposal/quote appears to be due to a lack of diligence or competence of the Vendor. However, the Government reserves the right to allow a Offeror to cure an informationally incomplete proposal/quote should the Government elect to request revised proposals/quotes or to request confirmation of the Government’s interpretation of information already contained in a proposal/quote (e.g., to rule out an ambiguity or potential vagueness) or for the Government to supply missing information that can be derived from other parts of the proposal/quote (confirming such with the Vendor). Offerors whose proposals/quotes simply claim compliance with requirements (e.g., contain statements that merely repeat the requirements of the statement of work (SOW) or agree to comply with those requirements), without explanation of how they will be performed, may be regarded as showing a lack of comprehension of the Federal Communication Commissions (FCC) requirement and may be removed from the competition or if permitted to remain will likely result in a negative finding.

EVALUATION FACTORS

The evaluation factors for this solicitation are included below in descending order of importance:

Factor 1: Implementation Capability

Factor 2: Relevant Experience

Factor 3: Technical Approach

Factor 4: Management Approach

Factor 5: Price

The order of importance among the factors are as follows: Factor 1: Implementation Capability is more important than Factor 2: Relevant Experience. Factor 2: Relevant Experience is more important than

Factor 3: Technical Approach. Factor 3: Technical Approach is more important than Factor 4:

Management Approach. Factors 1, 2, 3, and 4, when combined are significantly more important than Factor 5: Price.

Technical proposals will be rated using confidence ratings for Factor 1 and Factor 2, and adjectival ratings for Factor 3 and Factor 4. While Factor 5: Price will be evaluated, it will not be rated. Price will not be assigned an adjectival rating, or scored.

Factors 1 through 5 are described in detail below. While each factor will be principally evaluated by reference to the proposal volume or section on the same topic, the Government may also consider other related or interdependent proposal material in other volumes or sections. However, the Government may elect to provide a lower evaluation for proposals that do not include the information in the appropriate sections of the proposal volume as it is not the responsibility of the Government to search for and find the content in other sections or volumes of the proposal.

Factor 1: Implementation Capability

Factor 1 will be evaluated under a confidence rating schema that assesses the FCC’s determination and confidence in the Offeror’s ability for the implementation to meet and deliver on the requirements in the required timeframe and minimize risk to the Auctions Program. The following table provides confidence rating definitions.

High Confidence

The Government has a high expectation that the Offeror will preserve the integrity and functionality of the FCC’s Auctions Program. The Offeror convincingly demonstrates that it will meet or exceed the Government’s requirements in the requested timeframe, will provide a secure system through meeting technical and security requirements, and has a strong approach with sufficient explanation for accommodating requirements that cannot be met in a way or in the specified timeframe that minimizes Auction Program risk.

Confidence

The Government has a moderate expectation that the Offeror will preserve the integrity and functionality of the FCC’s Auctions Program. The Offeror demonstrates that it will meet the Government’s requirements in the requested timeframe, will provide a secure system through meeting technical and security requirements, and has a reasonable approach with explanation for accommodating requirements that cannot be met in a way or in the specified timeframe that minimizes Auction Program risk.

Low Confidence

The Government has concerns about the Offeror’s ability to preserve the integrity and functionality of the FCC’s Auctions Program. The Offeror demonstrates it will provide a secure system through meeting technical and security requirements.

The Offeror demonstrates that it will meet some of the Government’s requirements in the requested timeframe, and has insufficient details or lacks the ability in its approach to accommodate requirements that cannot be met in a way or in the specified timeframe that does not minimize Auction Program risk.

No Confidence The Government has no confidence that the Offeror will preserve the integrity and functionality of the FCC’s Auctions Program. The Offeror cannot provide a secure system through meeting technical and security requirements. Or the Offeror fails to meet a significant portion of the Government’s requirements in the requested timeframe, and has insufficient details or lacks the ability in its approach to accommodate requirements that cannot be met in a way or in the specified timeframe that does not minimize Auction Program risk. Proposal is not eligible for award.

The Government will evaluate the Offeror’s implementation capability presented in Factor 1 and its approach towards meeting the scope and objectives of the entire SOW and specifically the requirements identified in Appendix 1 Section A, B, and D. The Government will evaluate how the Offeror will execute the implementation, milestones for the implementation, and plans for transitioning to the new bidding system. The Government will evaluate the Offeror’s draft Implementation Strategy and Plan and the completed requirements template (including evidence provided by Offeror) to gain insight into the Offeror’s ability to successfully and timely meet the needs and support the FCC’s Auctions Program and its approach toward meeting the requirements of SOW and Appendix 1.

The Government will evaluate the Offeror’s Implementation Strategy and Plan and the completion of the requirements template (including evidence provided by Offeror):

(a) The ability of the Offeror to meet the minimum requirements of the auction formats required upon contract award

(b) The extent to which the Offeror can satisfy the minimum requirements in the timeframe set forth in the SOW

(c) The extent to which any trade-offs will be needed for any minimum requirements that may be identified as being unable to be accomplished in the timeframe set forth in the SOW, and the extent of the impact this may have on the FCC’s Auctions Program

(d) The extent the Offeror can meet the technical, hosting, and security requirements within the timeframe

(e) The Offeror’s approach and planning to meet the overall objectives to support the FCC’s Auctions Program

(f) The Offeror’s plan, creativity, and ability to meet future requirement(s) it will eventually be able to meet outside of the identified six month or one year timeframes set forth in the SOW

(g) The extent to which the Offeror’s Implementation Strategy and Plan aligns with the FCC requirements

(h) Offeror’s ability to describe and take into account issues, risks, or concerns as it relates to system implementation and integration (SOW: Activity 5) and the overall implementation strategy

(i) The feasibility of Offeror’s plan to make available a working bidding system in production for use by the FCC for a live auction

Factor 2: Relevant Experience

The Offeror's relevant experience will be evaluated based on the Offeror’s experience of a similar size, scope and complexity to the requirements listed within the SOW.

Each Offeror shall identify no more than three (3) recent and relevant projects with the United States Federal Government (preferred), or with State, local, commercial, or non-profit customers. “Recent” is defined as active (ongoing for at least 1 year) or completed efforts performed (with at least 1 year of performance) within the past five (5) years from the issuance date of this solicitation (this can include work that began prior to this five-year period but was completed within the five-year period). “Relevant” is defined as similar in terms of size, scope, and complexity in comparison to the requirements in the SOW. The recent and relevant demonstrated experience of subcontractors for this contract will be considered; however, at least two (2) projects must be of the Offeror serving as the Prime Contractor.

The Offeror shall describe the size, scope, and complexity of each project. The Offeror shall describe why the project is pertinent to the work covered under this SOW.

The Offeror shall provide the following information regarding each Project (at most two projects can be from the same contract):

• Project title

• Brief description of the project

• Company-specific role, responsibility, and deliverables

• Contract the project is under and, if applicable, call order number

• Contract/task value

• Type of contract/task (FFP, LH, etc.)

• Period of Performance for the contract/task

• Government Agency/Organization

• COR’s name, address, and phone number

• Contracting Officer’s name, address, and phone number

• Current status of the project (e.g., completed and/or, if in progress, start and estimated completion date)

• Relevance to the FCC’s requirement (where performance similarities make the project relevant, specifically identify by section those aspects of the SOW).

• Problems encountered and the Offeror’s corrective actions (if applicable)

• Outcome and delivery of the tasks/work/project

Confidence Rating for Factor 2

High Confidence

Extensive experience exists working on projects similar in size, scope and complexity to that contemplated by the solicitation. The highest quality of contract performance is anticipated with little or to no risk for disruption of schedule, increase in cost, or degradation of performance. The risk of unsuccessful contract performance is very low.

Confidence

Adequate experience exists working on projects similar in size, scope and complexity to that contemplated by the solicitation. There is a fair probability of success with a moderate degree of risk for disruption of schedule, increase in cost, or degradation of performance. Performance meets contractual requirements. The risk of unsuccessful contract performance is low.

Low Confidence Limited experience exists working on projects similar in size, scope and complexity to that contemplated by the solicitation. There is a low probability of success with high degree of risk for disruption of schedule, increase in cost, or degradation of performance. The risk of unsuccessful contract performance is moderate to high.

No Confidence Limited experience exists working on projects similar in size, scope and complexity to that contemplated by the solicitation. The offeror’s lack of adequate experience presents an unacceptably high risk of contract performance failure, and/or significant potential exists for disruption of schedule, increase in cost, or degradation of performance. The proposal presents an unacceptably high risk of performance failure. Proposal is not eligible for award.

Factor 3–Technical Approach

FCC will evaluate how efficiently and effectively the Offeror’s technical approach addresses each of the requirements of the SOW Section V. The Offeror shall describe their technical solution for completing the work activities described in Section V of the Statement of Work (SOW) and provide response/proposed solution for each Activity Area.:

Activity A: Contract Management

Activity B: Software, Maintenance, and Other Support

Activity C: Software Delivery Method - Hosting Services

Activity D: Technical Support (which includes Operations and Maintenance and Security Capability)

Activity E: System Implementation and Integration

Activity F: Special Projects

FCC will evaluate the merits of the Offeror’s technical approach to provide auction software and services as specified in this solicitation. Proposals will be evaluated on the basis of the Offeror’s demonstrated understanding of all elements of the software, technical and service requirements, the soundness and technical ability of the proposed software and technical approach and the validity of the proposed methodology to meet the requirements in the SOW as required in the solicitation.

Proposals will be reviewed and evaluated in accordance with the evaluation criteria identified below:

1. Understanding of the work and knowledge of technical requirements, including creativity and thoroughness shown in understanding the objectives of the SOW and specific activities/tasks, and planned execution of the effort.

2. Evidence of specific methods and techniques for completing and fulfilling each Activity, to include such items as risk management, quality control and customer-service.

3. Insight into and ability to explain and anticipate potential program challenge areas; major difficulties, problem areas, and risks; the application of creative and feasible solutions; future integration of new processes; and technology enhancements.

4. Degree to which the Offeror’s proposal demonstrates an understanding of logistics, schedule, and any other miscellaneous issues in which the Government should be aware.

5. The degree of assurance the proposal provides that the Offeror can and will successfully perform the required auctions services and operational and technical support, both during and between auctions, as hosted, and that the IT service level will be met.

6. Degree to which proposed approach will meet the performance standards and deliverables of the contract, including the quality of measures to identify, prevent, and ensure non-recurrence of defective services and employ best practices.

7. The identification and clarity of the cloud hosting model to be deployed, the comprehensiveness and rigor of data protections being offered from the initial planning stage through close-out, and the Offeror’s service guarantee or uptime of the software and associated cloud computing service.

8. The sufficiency and merit of the Offeror’s risk and failure prevention and mitigation plans and how they are apt to lessen impacts on a live auction that potentially threaten the objective of the work and potentially raise the Government’s obligations or liabilities due to system downtime or failure.

9. The detail and rigor of the Offeror’s security features and measures.

At a minimum, the offeror’s Technical Proposal for Factor 3 must discuss and will be evaluated the following:

1) The background, objectives, and work requirements of the statement of work as analyzed by the Offeror in their own words

2) The proposed methods and techniques for completing each Activity A, B, C, D, E, and F.

3) How the full performance, requirements, and needs within each Activity will be met by the

Offeror’s work;

4) Any anticipated major risks, difficulties and problem areas, along with recommended practical approaches for their resolution, and

5) Major logistical considerations.

When discussing each Activity, the following information should be addressed and will be evaluated:

A. Contract Management The offeror is expected to showcase their comprehensive approach for contract management. This includes outlining their strategies, methodologies, and procedures to effectively and efficiently handle all aspects of the contract throughout its lifecycle.

The demonstration of the contract management approach may include, but is not limited to, the following key components:

1. Contract Administration: How the offeror plans to manage day-to-day activities, ensuring compliance with contract terms and conditions, handling contract modifications, and addressing any issues that may arise during the contract performance.

2. Communication Plan: How the offeror intends to establish and maintain clear and effective communication channels with both the Government and any subcontractors or partners involved in the project.

3. Risk Management: The approach to identify potential risks, assess their impact on the contract, and develop mitigation strategies to minimize their negative effects on project execution.

4. Quality Control and Assurance: The offeror's plan to control and evaluate products and deliverables meet the required quality standards, comply with contractual requirements, and meet any agreed upon methods and metrics for measuring and monitoring performance.

5. Personnel Management: How the offeror plans to allocate and manage personnel resources to fulfill the contract's requirements effectively.

B. Software, Maintenance, and Other Support The Offeror is expected to showcase their comprehensive approach for the software, maintenance, and other support.

The demonstration of software, maintenance, and other support may include, but is not limited to, the following key components:

(a) Providing a license for the core auction bidding system service capable of meeting the scope and requirements of this SOW;

(b) Providing maintenance, patching, and software updates, included in the licensing approach, covered within the auction bidding software and service in support of the services outlined in the SOW;

(c) Providing technical support and documentation for the core auction bidding service; and

(d) Providing a software bill of materials (SBOM) and test for threats in compiled software binaries, in accordance with the National Institute of Standards and Technology (NIST) guidance on secure software development practices.

C. Software Delivery Method – Hosting The Offeror is expected to showcase their comprehensive approach for the software delivery method and hosting.

The demonstration of a software delivery method and hosting approach may include, but is not limited to, the following key components:

(a) Providing a cloud-based hosting service in support of the services outlined in the SOW.

(b) Providing the hosting service such that it will allow FCC the flexibility to meet future requirements.

(c) Providing all of the technology, architecture, and infrastructure that serve as the foundation for running the software application and the services on that infrastructure.

(d) Providing the software and services for running and managing access to the software and systems that transfer data or provide integrations with the FCC.

(e) Providing all software, hardware, and environments needed to develop, test, host, and use the service, that allows for the flexibility to quickly and easily establish and then shut down environments, areas, or instances to run an auction (whether in production or in a test scenario).

D. Technical Support The Offeror is expected to showcase their comprehensive approach for technical, IT security, and operations and maintenance (O&M) support.

The demonstration of an approach to technical, IT security, and O&M support may include, but is not limited to, the following key components:

a) Provide all O&M for the hosting service and all systems within the hosted service, to include all IT security related updates and reviews and technical service support.

b) Provide ongoing support for the operations, maintenance, and security (in accordance with FCC Security Policy and federal guidelines) of the deployed system infrastructure.

c) Provide reviews, analysis, and documentation as needed for required assessments, such as for privacy and data

d) Provide a system that is able to obtain an Authority to Operate (ATO) and produce documentation in support of an annual review and /continuous monitoring process to maintain an ATO.

e) Provide service support – from basic end-user application support to resolution of complex software issues.

f) Ensure that the service and all systems maintain the necessary uptime and that a proper Continuity of Operations plan is in place that can be executed if needed.

E. System Implementation and Integration The Offeror is expected to showcase their comprehensive approach for making the bidding system available and integrated with the FCC technical domain and customizing/configuring the software to meet the requirements as defined in the SOW. The Offeror may reference Factor 1 – Implementation Capability.

The Offeror shall demonstrate a system implementation and integration approach that shall include, but is not limited to, the following key components:

a) Its approach to ensure that the system is available to establish the connections to the FCC network/firewall and integrated systems, and configure and modify the system to meet minimum requirements that will allow the FCC, after the necessary FCC testing, to use the service auction bidding system for a live auction.

b) Its approach to support all testing activities, testers and test teams, and perform remediation on issues found or items determined to not sufficiently meet the requirements such that the FCC will be unable to use the service in the production environment for a live auction. Include information on what the offeror does or has in the system to ease the use/burden for creating and running automated testing by the FCC’s contracting partners.

c) Its approach to develop a system training plan and system training materials, and deliver the system training.

d) Its approach and coordination efforts to deploy the system into a production environment and make available for the FCC’s use.

e) Its approach to create and provide detailed documentation such as functional and technical requirements, software configuration/modification documents, user manuals, and operating procedures,.

f) Its approach to Project Management in support of the implementation and integration.

F. Special Projects

The Offeror shall demonstrate its approach to:

a) Be responsive to meet and flexible to implement the FCC’s future auction and bidding system needs for modifications or new auction formats or bidding system functionality

b) Ensure auction design and economics of auctions consultation and advisory services can be provided by auction experts who hold a PhD in economics, operations research, or other closely related field

c) Provide additional technical, configuration, or operational services in support of conducting live auctions, including mock auctions

Adjectival Rating for Factor 3:

Superior

The proposal outlines an effective, efficient, and achievable approach for meeting requirements as defined in the SOW. The proposal’s strengths significantly outweigh any weaknesses, if any, in a manner that will significantly benefit the government. There are no significant weaknesses and no deficiencies. The risk of unsuccessful contract performance is very low.

Good outweigh any weaknesses in a manner that will benefit the government.

There are no significant weaknesses and no deficiencies. The risk of unsuccessful contract performance is low.

Satisfactory

The proposal outlines an achievable approach for meeting requirements as defined in the SOW. The proposal’s strengths, if any, are balanced out by weaknesses. There are no deficiencies. The risk of unsuccessful contract performance is low to moderate.

Marginal The proposed technical solution outlines a marginally achievable approach for meeting requirements as defined in the SOW. The proposal’s weaknesses outweigh any strengths. The proposed solution has no deficiencies. The risk of unsuccessful contract performance is moderate to high.

Unsatisfactory The proposal fails to demonstrate an achievable approach for meeting all the requirements defined in the SOW. The proposal contains a deficiency or multiple significant weaknesses that present an unacceptably high risk of performance failure. Proposal is not eligible for award.

Factor 4: Management Approach

The FCC will evaluate the extent to which the Offeror’s proposed management approach demonstrates a clear understanding of all proposal/quote requirements through demonstrated capability and experience in managing projects that meet project objectives, within budget and on schedule.

The FCC will evaluate the Offeror’s proposal to determine the extent to which its proposed models/processes/tools are likely to be successful in gaining efficiencies and/or process improvements regarding adequacy and availability of proposed personnel and facilities to perform program and project tasks.

This Factor will also include the initial draft of:

• the Program Management Plan (PMP), including the Staffing Plan

• the Quality Control Plan (QCP)

FCC will also evaluate the extent to which the Offeror understands the requirement based on clarity, logic, and effectiveness of the program/project organization and its proposed labor mix (Key Personnel and other proposed staff) and level of effort (number of staff) for firm fixed price (FFP) and labor hour (LH) CLINs.

The FCC will evaluate the quality of Offeror’s management approach including the credibility and resilience of the approach.

FCC will evaluate the quality of the Offeror’s proposed approach to staffing, retention, and recruiting/screening policies for this requirement.

The Staffing Plan shall ensure the FCC receives well-qualified Offeror personnel, including capabilities and experience of personnel related to this work, and the proposed methodology for ensuring that the most suitable and best employees are engaged to perform throughout the term of the requirement.

FCC will evaluate the Offeror’s allocation of personnel and resources based on key personnel requirements outlined in SOW Section V.D. The Government will evaluate the qualifications, demonstrated abilities, and resumes of the identified Key Personnel.

FCC will evaluate the quality control approach to ensure the services, deliverables and management of the work under this contract will effectively ensure quality of what the FCC receives throughout the contract period of performance, and systematic methods can be used to assess the Offeror’s performance and delivery of services and products.

Adjectival Rating for Factor 4:

Superior significantly outweigh any weaknesses, if any, in a manner that will significantly benefit the government. There are no significant weaknesses and no deficiencies. The risk of unsuccessful contract performance is very low.

Good outweigh any weaknesses in a manner that will benefit the government.

There are no significant weaknesses and no deficiencies. The risk of unsuccessful contract performance is low.

Satisfactory

The proposal outlines an achievable approach for meeting requirements as defined in the SOW. The proposal’s strengths, if any, are balanced out by weaknesses. There are no deficiencies. The risk of unsuccessful contract performance is low to moderate.

Marginal The proposed technical solution outlines a marginally achievable approach for meeting requirements as defined in the SOW. The proposal’s weaknesses outweigh any strengths. The proposed solution has no deficiencies. The risk of unsuccessful contract performance is moderate to high.

Unsatisfactory The proposal fails to demonstrate an achievable approach for meeting all the requirements defined in the SOW. The proposal contains a deficiency or multiple significant weaknesses that present an unacceptably high risk of performance failure. Proposal is not eligible for award.

Factor 5: Price

Price proposals/quotations will be evaluated by the Contracting Officer based upon the total price reflected in the completed Pricing Sheet, including all options. Evaluation of the option under FAR 52.217-8, Option to extend services, will be accomplished by using the pricing offered for the last option period. For evaluation purposes only, fifty percent of the cost offered for the last option period will be added to the base and all other options to arrive at a total price. Price will be evaluated for price reasonableness using any of the relevant techniques described in the Federal Acquisition Regulation.

The ceiling amount for the Special Project CLINs will be included for price evaluation.

BASIS FOR AWARD

The Government will award the contract to the responsible Offeror whose proposal, conforming to this solicitation, represents the best value to the Government after evaluation in accordance with the evaluation factors stated in this solicitation. The proposal that offers the best value is the proposal that, in the Government's estimation, provides the greatest overall benefit to the Government in response to the requirement. In making this determination, the Government is more concerned with obtaining superior technical/management capabilities than with making an award at the lowest price. However, the Government reserves the right to award the contract to an Offeror whose proposal offers other than the lowest price or whose proposal does not offer the most superior non-price attributes, where doing so achieves the greatest overall benefit for the Government. In determining which proposal offers the best value, the Government may make trade-offs between prices and non-price proposal attributes.

Receiving less than a satisfactory rating in any single factor will result in the overall proposal being determined not acceptable, and therefore ineligible for award.

The Government intends to make award without further communicating with Offerors. Offerors are highly encouraged to quote their best technical and price quotes in their initial submissions. However, the Government reserves the right to communicate with any or all Offeror’s submitting a technical and price quote. An Offeror may be eliminated from consideration without further communication if its technical and/or pricing quotes are not among those Offerors considered most advantageous to the Government based on a best value determination. Proposals which merely reinstate the requirement provided in the solicitation will also not be considered eligible for award.

Proposals shall set forth full, accurate, and complete information as required by this solicitation package (including Attachments). The penalty for making false statements in proposals is prescribed in 18 U.S.C.

1001.

The Government defines significant strength strengths, weakness, significant weakness, and deficiency in its evaluation of proposal for Factor 3 and Factor 4 as follows:

Significant Strength: An attribute of the proposal that appreciably enhances the merit of a proposal or appreciably increases the probability of successful contract performance.

Strength: An attribute of the proposal that approaches a requirement of the solicitation in a way beneficial to the Government or in a way that increases the likelihood of successful contract performance.

Weakness: A flaw in the proposal that increases the likelihood of unsuccessful contract performance.

Significant Weakness: A flaw in the proposal that appreciably increases the likelihood of unsuccessful contract performance.

Deficiency: A material failure of the proposal to meet a requirement or a combination of significant weaknesses that increase the likelihood of unsuccessful performance to an unacceptable level.