08_DIR_T F_ATTACHMENTASOW_12-18-07.doc

ATTACHMENT A

FY 2008 HRSA Teleconference and Field Objective Review Committee (ORC) Meetings Statement of Work

12/18/2007 I. Background HRSA receives about 6,000 applications per year for over 100 separate programs. Each eligible application must be evaluated by a minimum of three individuals (review participants) who have knowledge and expertise in the program area under review. Each reviewed grant application must be individually scored. Review participants selected for service must first certify they have no known or apparent conflicts of interest. Each applicant receives a review summary which cites the major strengths and weaknesses of their application as determined by the panel. Information gathered and transmitted through this process must be quality controlled (QC) at designated steps to assure that all applications are objectively reviewed, scored and recorded without error. In 2007, approximately 100 reviews were held using meetings (45%), teleconferences (45%), and field reviews (10%).

All tasks in this Statement of Work (SOW) are completed for an Objective Committee Review (ORC) that are either web-assisted Teleconference meetings or Field reviews (T&F). Field reviews (Field) perform all web-assisted teleconference SOW tasks substituting an administrative collation of reviewer evaluations for the meeting, and conducting a consensus teleconference when requested by DIR.

In teleconference meetings, the review participants receive, read, and independently score their assigned applications in advance of the meeting. Review participants then meet in a web-assisted teleconference for a group discussion, prepare summary statements of the application significant strengths and weaknesses, assign a score, and may vote when required. Depending on the number of applications under review, a program review may consist of multiple review committees called panels. The Summary Statement is the final product of the review process and must be an accurate reflection of the significant strengths and weaknesses of the application, and support the ranking score or funding recommendation(s). As a general overview of contract duties, the Contractor is expected to:

Assist HRSA in securing and assigning review participants into review groups called panels

The pick up, inventory, assignment, and mail out of review materials

Arrange travel, lodging, and reimbursement for review participants as necessary

Arrange and pay for meeting logistics

Deliver final electronic and paper copies of documents to HRSA after the meeting Use general and specific security measures as appropriate to safeguard government information, documents, and equipment used for review meetings.

At DIR’s discretion, for rare exceptions and under extenuating circumstances, one or more of these meetings may be a “tiered” review, i.e. a combination of Web assisted Teleconference, Field Reader Reviews, or collaborative meeting. If the type of review is changed to a face-to-face, contract costs and duties shall be negotiated and adjusted at that time.

Some examples of exceptions:

a. Delayed deadlines requiring a collaborative meeting to expedite the review

b. To include an application previously determined to be ineligible

c. The number of applications received is significantly less than anticipated

In the collaborative review process, each application is assigned to three members of the ORC who serve as principal review participants and who form a “collaborative triad” for those applications. Each reviewer will receive an on-site review package: 1) his/her triad and application assignments; 2) extra copies of the worksheet; and 3) short sets (a short set consists of the application face page, abstract/summary, budget pages, and budget justification).

Each triad is provided a laptop computer as well as a copy on electronic media, i.e. compact disc (CD) or flash drive, for preparing documents prior to the panel review meeting. The media copy contains Worksheet templates; one labeled for each grant application the team is assigned. Review participants will not receive the applications (either paper copies or CDs) prior to the meeting dates. Instead, they will receive paper copies of the applications upon arrival at the meeting site. When inviting review participants and noted on the Request for Task Order (hereby known as Request for Services (RFS) form), the Contractor should encourage them to bring a laptop if they have one in order to assist with their work as individuals and in the triads. Materials mailed-out include a copy of Program Guidance, Chair/Reviewer Instruction Guides, and travel and hotel arrangements. Review participants should be instructed to bring the mail-out material to the review meeting.

Teleconference and Field meetings do not generally travel or convene review participants in person. Teleconference reviews may travel chairpersons to HRSA for the review meeting. In Teleconference reviews, review participants evaluate grant applications independently and HRSA convenes a panel discussion using the telephone, Internet, or both. In Field reviews, individuals review applications off site and submit their assessments in writing to HRSA with no panel discussion. When the Teleconference review is convened, the review participants shall initially review, score, and then present their detailed assessments of the application to the panel. Based on panel consensus, the Contractor shall prepare summary statements of the application significant strengths and weaknesses, and include the panel scores and voting as necessary.

Teleconferences may be web-assisted, meaning that the review participants would have a computer link enabling them to see the draft summary statement as it is being created and edited in real-time during the telephone discussion of the application. The Internet component of a web-assisted teleconference will be paid for by HRSA, and handled by HRSA’s Office of Information Technology (OIT). The Contractor shall coordinate the web-assisted teleconference review with HRSA-OIT accordingly. It is anticipated that approximately 90% of the proposed teleconferences shall be handled in this manner.

For Field reviews, review participants are to have 30 days to evaluate applications and submit competed worksheets or checklists back to the Contractor. In cases where there is a clear lack of consensus, phone calls may be placed to individual review participants, a panel teleconference held, or other meeting types employed to resolve the divergent evaluations. It is then projected that HRSA, with the assistance of the Contractor, will have 27 days to prepare final summary statements after receipt of the three review participants worksheets, calculate mean scores, and generate the rank order list.

The Contractor is defined as the company or legal entity and their designee, employee, or representative, and does not include HRSA personnel. The Contractor shall provide administrative and logistical support, including non-federal review personnel on site, to the Division of Independent Review (DIR) Staff Lead who is assigned as the Federal Review Administrator (Federal RA) to a particular review during the planned Objective Review Committee (ORC) meetings. Historically, a typical teleconference meeting ran 3 days and involved approximately 20 review participants (including chairs) assigned to 3 different panels, reviewing 37 applications. Historically, a typical field review meeting involved approximately 23 review participants, reviewing 30 applications. The actual numbers of applications, review participants, and panels shall vary as grant applications are received by HRSA .

The number of review participants required for an objective review depends on the following:

· Number of anticipated and/or actual applications to be reviewed

· Number of review participants to be assigned to review each application

· Number of applications to be assigned to each reviewer

ORC’s may be scheduled during any month of the year, but most often occur in the second and third Quarters of the fiscal year. The Contractor must be able to plan the workload and provide qualified staff within narrow time frames to complete the contract requirements. Expenses incurred in support of ORC meetings include (among others): contractor salary and overhead; reviewer travel, lodging, and honoraria; shipment of review packages; and meeting facility costs. Deliverables include an Official Review File, a Summary Statement for each grant application that was reviewed, a Rank Order List, and reviewer reimbursement.

Regulatory Compliance:

1. Contract performance, including the prevention of a conflict of interest, shall be in accordance with the:

a. Awarding Agency Grants Administration Manual, Chapter 2.04.104C “Objective Review of Grant Applications”

b. HRSA Grants Policy Directive Pre-Award, “Awarding Grants” (Revised 11/17/99),

c. The DIR Objective Review Manual

2. The Contractor shall comply with the applicable Federal guidelines for the protection of privacy:

a. Privacy Act of 1974

b. Health Insurance Portability and Accountability Act (HIPAA)

c. Freedom of Information Act (FOIA)

d. OMB M-06-15: Safeguarding Personally Identifiable Information

e. OMB M-06-16: Protection of Sensitive Agency Information

f. OMB M-06-19: Reporting Incidents Involving Personally Identifiable Information Incorporating the Cost for Security in Agency Information Technology Investments

g. OMB M-06-20: FY2006 Reporting Instructions for the Federal Information Security Management Act and Agency Privacy Management

3. The Contractor shall follow the applicable Federal Guidelines on computer security, data protection, record retention, and confidentiality listed below. All Contractors, staff and authorized users of HRSA’s IT resources must complete the Security Awareness Training and Standards of Conduct, and must read and sign HRSA’s Rules of Behavior. This requirement must be completed within 30 days of being granted access), and again annually for the duration of the contract.

4. Federal law also dictates that users must complete an annual refresher course designed according to their security responsibility.

a. Computer Security Act

b. Presidential Decision Directive (PDD)-63

c. Federal Information Security Management Act (FISMA)

d. OMB Circular A-130

5. Travel and reimbursement shall be made in accordance with the HHS and Federal Travel Regulations (FTR).

6. The HRSA objective review process must meet the highest ethical standards. HRSA and the Contractor must avoid any circumstance that might introduce any actual, potential, or the appearance of any conflicts of interest (COI’s), prejudices, biases, or predispositions into the process. A reviewer of a grant application may not have a relationship with the applicant organization or any personal or vested interest in the award of a grant to that organization as defined in AAGAM 2.04.104C-6C.6. As provided in AAGAM 2.04.104C-6A.5 and A.6, when selecting qualified review participants, HRSA and the Contractor must ensure that no two members of the same review panel or group of field readers are from the same organization or institution. Title 45 Part 73 of the CFR contains waivers for certain multi-campus institutions, allowing them to be treated as separate organizations for purposes of determining whether an individual’s employment creates a COI.

II. Purpose The purpose of this contract is to provide Administrative, Technical and Logistical support services to the Health Resources and Services Administration (HRSA), Office of Federal Assistance Management (OFAM), Division of Independent Review (DIR) to: plan, direct, and execute the independent review of applications for HRSA grants and cooperative agreements for web-assisted Teleconference meetings and Field reviews. The objective review process is designed to be ethical and professionally competent. Reviews must be conducted in a timely manner, without conflicts of interest, and be conducted in a professional manner even in unforeseen situations. Review results must be substantively and editorially accurate.

III. Overview This Statement of Work (SOW) is for Objective Review Committee (ORC) web-assisted teleconference meetings and field reviews where the review participants receive, read and independently evaluate their assigned applications in advance of the meeting. Meetings are conducted as web-assisted teleconferences to discuss, score, and prepare summary statements. Collaborative and Teleconference meetings are conducted in panels with a Chair to discuss, score, and prepare summary statements. Field reviews are directed by HRSA, conducted independently by the review participants who send their worksheets by email to the Contractor. In all reviews, the Contractor merges the worksheets into draft summary statements. In the case of Field reviews, HRSA is the final review and approval for summary statements. Where used, the Government Project Officer shall also mean Project Officer (PO), and the Government Contract Officer shall also mean Contract Officer (CO).

Except for travel and lodging, all of the steps in the Statement of Work are performed by the Contractor for a teleconference and web-assisted teleconference meeting. For Field reviews, all of the teleconference steps apply except the panel meeting. Services for each review are requested using a Request For Services (RFS) form. All charges and activities shall be ordered, tracked and reported by the HRSA announcement number in all correspondence and reports.

The Division of Independent Review (DIR) Staff Lead is assigned as the Federal Review Administrator (Federal RA). The Contractor shall provide administrative and logistical support, including non-federal review personnel to support the Federal RA in a particular review during the planned Objective Review Committee (ORC) meetings. DIR expects the Contractor to work in accordance with the HRSA/Office of Federal Assistance Management (OFAM) /DIR Objective Review Manual. Unless otherwise waived, the Contractor shall use the DIR proscribed forms for correspondence and data management, and shall use a standardized DIR file naming convention. Updates to the DIR Manual and forms shall be available to the Contractor as they are posted to a shared web site. HRSA and DIR require the use of PC based products to include programs for word processing, data tables, databases, graphic presentations, email, project planning, and statistical packages. All electronic files shall be in ClarisWorks (CWS) compatible format or comma separated files. Electronic media should be used to the maximum extent possible to carry out objective reviews. Electronic media include e-mail, facsimile machine, flash drives, compact disks (CDs), Internet and video conferencing.

IV. Tasks:

In performance of this contract, the Contractor shall perform the following tasks for each review meeting:

A. Pre-Review Planning:

Pre-Review Activities are required for each planned review, including selecting qualified review participants; receiving and processing applications; confirming the review approach; making logistical arrangements; assigning applications to individual review participants; preparing and transmitting review materials to review participants; assigning review participants to panels; informing review participants of confidentiality requirements; and conducting a pre-review orientation with review participants.

Training of Contract Staff. Following orientation and initial training by DIR staff, the Contractor shall conduct such training as necessary to ensure staff preparedness to dependably function in the roles outlined above. DIR shall conduct annual refresher training as needed or by request.

All deliverables, tasks, personnel requests, and performance dates or schedules shall be specified by the RA to the Contractor by review type on the DIR Request for Services (RFS) form (example in Attachment B). Where not specified in the RFS, task performance is understood as specified in the SOW.

Task 1(a) Government Meeting with Contractor.

This task covers the initial post-award contract meeting and the first meeting of each option year. The Contractor shall meet with the DIR Project Officer (PO) within (5) five days of the effective date of the contract (EDOC). The meeting shall take place at the office of the PO to discuss the scope of the contract, including: goals and objectives, processes, time frames and schedules, training needs, priorities, site requirements, travel, logistics, general orientation to relevant aspects of the HRSA/DIR database, and other related activities of the contract. The Contractor shall provide the PO a contract staff roster with contact information and position title that is updated quarterly, or when there are staff changes. This is in addition to the staff manning roster and contact information provided for each review assignment.

Based on the discussions at the initial PO meeting, the Contractor shall submit, within (5) five days after the initial meeting with the PO, assignment of contract personnel based on the schedule of reviews, a review checklist and preliminary plans, with anticipated deadlines for the first month’s reviews. Subsequent ORC reviews shall be planned and coordinated using the timelines as outlined in the DIR Objective Review Manual.

Task 1(b) Pre-Review Functions - General Overview.

During the period of performance of this contract, contract staff must be able and available to complete the contract requirements at, or above, the level of satisfactory standards. In advance of critical deadlines and review meetings, the Contractor shall: notify the PO of any perceived conflicts in workload or scheduling that may jeopardize full task performance and review completion; provide rational solutions with revised cost estimates; re-negotiate delivery dates with the Federal RA and the PO; and if applicable, devise a strategy to prevent future recurrence. The Contractor shall maintain clear documentation for the order, receipt, or delivery of key material, data, or documents from Federal RA’s required to complete the contract tasks and include this data in the weekly written progress report. The Contractor shall provide to the Federal RA contact information for staff assigned to the review within 24 hours after receipt of the initial Task Order. When a courier is either picking-up or delivering materials to the Parklawn Bldg., the courier shall contact a DIR Policy staff person at least 4 hours before and immediately after arriving in the DIR office Task 1(c) Pre-Review Functions - Progress Reports.

The Contractor shall submit a weekly written progress report. The progress report content, data format, and due date shall be proscribed by DIR with a general description noted in section V. Schedule of Deliverables. The report form shall be provided to the Contractor after award.

Task 1(d) Pre-Review Functions - Logistics.

Where possible, HRSA facilities will be used for teleconference meetings. If HRSA facilities are not suitable or available, the Contractor shall submit not less than (3) three possible non-Government sites for lodging and/or meeting space in the Rockville, Maryland area (proximity to the Parklawn Building), or Washington, D.C./Metro Area (proximity to the Metro) to the PO for selection. Costs for suggested sites must be within or below the Federal per diem rate for specified areas. If required for meetings, the Contractor shall provide transportation or courier services for documents and an average of 20 boxes of grant applications from the Parklawn Building to their location, and /or the review facility in the DC Metro area; arrange for lodging for all panel members, and arrange for a meeting/conference room for each panel at the meeting site. The RFS shall specify the options for room amenities as part of the “facility package” cost. Unless otherwise specified, each meeting room shall include a central table to accommodate approximately fifteen (15) panel members, and seating around the periphery for at least 4 HRSA staff members and Contract staff. The provision of any meals shall conform to the current DHHS food policy.

Overall logistical capability shall include graphics and printing resources or capability.

Financial accountability in logistical planning shall include:

a. Avoiding improper billing (i.e. excessive telephone charges and teleconferences when billed to a field review);

b. Infrequent use of one time charges of durable and reusable supplies (i.e. signs and supplies);

c. Early facility contact, aggressive cost negotiation, and contracting of lodging, meeting rooms, and amenity packages in advance of peak workload times to contain costs and provide better location accessibility for attendees.

d. A cost control and reduction plan for review participant travel to recoup or reuse cancelled and unused travel tickets.

e. Accurate tracking of credits and less-than-projected expenses; for example: lower final costs for hotel meeting rooms from the estimate; final lower charges for amenities; final lower reviewer travel charges; accounting methods show how the credits are carried accurately to the final statement; analyze and report which categories have the greatest margin of unpredictability.

Task 1(e)

Pre-Review Functions – Reviewer Travel and Reimbursement.

The Contractor shall conform to the HHS Federal Travel Regulations (FTR) and arrange for common carrier tickets and lodging for all panel members who qualify for Government travel to meetings, and mail, e-mail, or fax itineraries at least two weeks prior to the ORC. The Contractor must always seek the lowest possible fares within the specified parameters. In cases where the reviewer would prefer other arrangements than those offered by the Contractor, review participants can make the alternate arrangements but shall only be reimbursed to the Government’s cost as originally offered. The Contractor shall ensure that travel arrangement providers emphasize to meeting participants that tickets are non-refundable and changes shall result in an additional fee which shall be the reviewer responsibility to pay. The Contractor shall retain documentation that acknowledges that the participant understands/accepts these travel restrictions.

The HHS Travel Policy is found at:

· http://www.knownet.hhs.gov/log/AgencyPolicy/HHSTravelPolicy/travel_policy.htm The General Services Administration (GSA) FTR is found at:

· http://www.gsa.gov/Portal/gsa/ep/contentView.do?contentId=14161&contentType=GSA_OVERVIEW In cases where a medical exception for travel (First Class, train or auto) can be properly documented, the waiver request and approval shall conform to the HHS Federal Travel Regulations (http://www.hhs.gov/hhsmanuals/Travel_Manual.pdf ). Special needs travelers are processed under the Federal Travel Regulations, Title 41 of the Code of Federal Regulations (CFR), Chapter 300-304, Part 301-13—Travel of an Employee With Special Needs (Authority: 5 U.S.C. 5707). The Federal Travel Regulations are found at the GSA web site at: http://www.gsa.gov/Portal/gsa/ep/channelView.do?pageTypeId=8199&channelId=-14863. The Contractor must be prepared to provide local transportation from reviewer lodging to the meeting facility if such transportation is not provided by the hotel or HRSA.

The FTR defines Per Diem Allowance reimbursement for Meals and Incidental Expenses (M&IE) based on the following daily duty clock:

· More than 12 but less than 24 hours = 75 percent of the applicable M&IE rate.

If travel is 24 hours or more:

· The day of departure 75 percent of the applicable M&IE rate.

· Full days of travel 100 percent of the applicable M&IE rate.

· The last day of travel 75 percent of the applicable M&IE rate.

Task 1(f) Pre-Review Functions – Potential Reviewer Selection.

The Federal RA will provide the Contractor with names and contact information for potential review participants. This participant list is extracted from the HRSA Electronic Handbook (EHB) database using program appropriate coding. Participants must be listed in the database and screened by DIR and Program for the review before being contacted or accepted as a reviewer. The Contractor shall, within 5 days of receipt of the list of potential review participants, contact potential review participants and alternates to ascertain interest, availability, and maintain contact with selected review participants at all stages of the review. The Contractor shall generate a list of available participants for the review and send this list to the Federal RA with participant status updates on a weekly or other regular basis. Communication methods shall include email and web access, to include a tracking method of updates and planned serial communications to review participants (with a carbon copy to the Federal RA) and keeping a record of these contacts).

Every 2 weeks [after the “interest & availability” email is sent and until mail-out], the Contractor shall send an email to all participants who have responded as being available for the review. The purpose of this communication is to confirm participants who are still available, and brief them on review planning.

The email shall read: “The purpose of this email is to confirm that you are still available to participate in the (program name) review scheduled for (insert meeting dates). If your availability has changed, please let us know. DIR is continuing to complete review plans and will soon finalize reviewer selections. Please watch for future communication.”

Task 1(g) Pre-Review Functions - Reviewer Selection and Panels.

Federal RA will review and approve the participant availability list from 1(f), reviews the availability request letter with the Contractor, and approve the letter for use. When available, a list of program specific grant applications will be provided to the Contractor from the HRSA database by the Federal RA within 7 business days of closing the application acceptance date. Where directed, the Contractor shall fax, email or mail to the DIR approved participants the reviewer acceptance letter, COI forms (See Regulatory Compliance citations for the definition of conflicts of interest) and the list of grant applications. Where directed, the Contractor shall receive the COI forms from participants before the mailing of review materials. The Contractor shall keep the original COI in the Official File and route a copy to the Federal RA within 3 days of receipt. The Federal RA will review the COI forms and approve the available participants for use in panel selections and application assignments. Requests for COI waivers shall be sent to the Director of DIR for approval.

Upon receipt of the participant notice of availability to review and the COI certification, the Contractor shall create a draft roster of participants and alternates for each review meeting. The Federal RA and Program will review the draft roster and when approved, direct the Contractor to create draft review panels. In coordination with the Federal RA and Program, the Contractor shall make draft reviewer panel assignments, draft application assignments, begin preparations for the pre-review orientation teleconference, and release alternates when directed to do so. Unselected participants shall be notified by email within 3 days of the Federal RA notifying the Contractor. The Contractor shall collect and update reviewer data in HRSA’s database (reference task 4(b)).

HRSA demographic distribution requirements must be considered when creating panel assignments. Unless otherwise directed and waived, no first time HRSA reviewer shall be assigned as the first or second review presenter on the first day of the meeting. The Federal RA will have final approval of reviewer panel and application assignments. The Contractor shall email to the Federal RA a final participant list and panel assignments no more than one week after the mail out, and no less than one week prior to the meeting.

Task 1(h) Pre-Review Functions - Reviewer Correspondence.

Reviewer eligibility and Conflict of Interest (COI) forms, and a list of grant applications are sent to the review participants. The COI forms are returned to the Contractor who sends copies to the Federal RA. In cases where the COI forms are returned to the Contractor before the mail out, they are reviewed and approved by the Federal RA before the shipment of the application packets for review. All reviewer-submitted COI forms shall be reviewed and approved by the Federal RA before final panel and application assignments are made. The Federal RA will send to the Contractor a list of eligible applications approved for the specific program review. Review participants shall in turn receive in the mail out only eligible applications for their assigned panel. A short set consists of the application face page, abstract/summary, budget pages, and budget justification. The shipping and mailing shall include a tracking method to ensure proper and timely delivery of reviewer correspondence.

The Contractor should notify the Federal RA no less than two weeks in advance of the package mailing of any missing COI’s and follow up as requested by the Federal RA. Prior to the orientation telephone conference call and /or the panel meetings, the Contractor shall priority mail, email, or fax to each reviewer packet(s) of information per the RFS.

Review Participant’s Mail-out Package always contains a CD with the following information by panel:

Program Guidance DIR Instruction Guides for Reviewer or Chair as appropriate and by meeting type Template of the reviewer worksheet

All applications for that panel, complete document (no Short Sets) Hard copy documents to be included:

Complete Applications for primary, secondary, and third review participants (no “short sets”)

Reviewer Worksheets (one hard copy)

List of grant applications for review by panel with the Institutions and Project Director’s as listed on the applications Reviewer Application Assignment list

Specifics about pre-review orientation call

Teleconference Review connection instructions (web & telephone) as applicable

If applicable, ground transportation instructions shall be included in the package. All mail out documents listed above and noted in the DIR Request For Services (RFS) shall be reviewed and approved by the Federal RA before collation and shipment by the Contractor. The Contractor is responsible for: pick up, inventory, copying, repackaging, and delivery of the specified information to each reviewer.

Reviewer worksheets must be sent electronically to review participants for ease of use. Review participants return the completed initial reviews (draft Summary Statements) to the Contractor before the meeting. Using the RFS dates as a guide, the Contractor shall send reminder notices via email to review participants with a carbon copy email to the Federal RA in the event of any late submissions of worksheets or COI forms from review participants. The review participants email their completed Objective Review Worksheets (individual draft summary statements) to the Contractor. The Contractor then consolidates these drafts into a combined draft Summary Statement to be used in the panel discussion and modification at the meeting (See Task 2(e)). Package mailing shall use the most expeditious means with a tracking system to ensure receipt.

After the Mail-out:

Conduct the Pre-Review Orientation Teleconference

Maintain contact with participants on process updates and changes

Receive Worksheets from Review participants (bring the original COI forms and worksheet copies to the review in the panel file)

Notify review participants of worksheet receipt

Consolidate worksheets into Draft Summary Statements

Generate the “Order of Review” list of applications, a panel agenda from this list, and provide a draft copy to the RA for approval up to one month and no less than one week before the meeting.

Task 1(i) Pre-Review – Review Orientation Teleconference.

Contractor staff shall schedule and arrange one or more teleconference calls for review participants and/or chairs to occur within four (4) work days after the mail out of review materials and applications. The Federal RA will provide the teleconference agenda to the Contractor within 2 weeks of the meeting date. The Contractor shall: set up the teleconference, record the teleconference for later use, record the names of callers to both live and recorded sessions, distribute the agenda to all meeting participants prior to the call, and answer travel and reimbursement questions. When the Contractor is notified of schedule changes or meeting cancellations by the Federal RA, any billable meeting venue (teleconference or web assisted program) shall be cancelled by the Contractor no less than 26 hours before the scheduled meeting time and date to avoid unnecessary charges to HRSA. Failure on the part of the Contractor to execute a timely cancellation (with proper confirmation of the cancellation) shall not be reimbursed by HRSA without full documentation of the events preventing such timely cancellation. Unless another orientation method is selected, two weeks before the review the Contractor shall provide the RA with a list of names of the callers who attended the live and recorded teleconference sessions.

The Chairs shall have a separate orientation session. A separate new reviewer orientation session shall be conducted immediately after the pre-review orientation teleconference. All orientations shall be recorded and available for reviewer reference before the review meeting.

B. Review:

Conduct of the Review addresses the responsibilities of review participants; the review process in order of events; specifies the processes for evaluating and scoring individual applications; and indicates the requirements for evaluating the review process and reviewer performance. General review steps include: preliminary reviewer assessments; review of applications; scoring of applications and determining eligibility for funding factors; budget discussion, proposed award conditions, and recommendations. Contract personnel duties shall be staffed, conducted, and evaluated as outlined in the Statement of Work (SOW) and the Performance Evaluation metric of the contract. Only the Chair, or a HRSA approved reviewer as substitute, can conduct the review meeting. Reviews shall be conducted with a minimum of disturbance, interruption, or movement within the meeting room. All cell phones shall be set to vibrate, silent ringers, or turned off during the meeting. Unless pre-arranged, review participants shall answer calls only during breaks and meals.

Task 2(a) Review – Logistics.

At least five business days before all meetings, the Contractor shall provide the RA with contract team names and assignments participating in the review. Each panel must have the member COI forms and original reviewer worksheets available during the review. On the first day of the review, the Contractor staff shall meet with the Federal RA for a debriefing before the Reviewer Orientation presentation and the start of the meeting. A review panel room can be used for this meeting. There shall be a reviewer’s orientation on the morning of the first meeting day.

Where appropriate, Teleconference meetings shall use Government FTS conferencing services and other Federal facilities or utilities to conduct the review. Web-assisted meeting programs and accessibility shall be generally applicable to accommodate the various review participants’ skills and equipment.

For meetings not in the Parklawn Building, the contractor shall use a silent communication system to summon support staff outside the meeting room for assistance in panel rooms, for facility services, or to communicate with other staff.

The Contractor shall arrange for the review participants’ registration at the ORC meeting and provide:

· adequate staff to ensure smooth onsite check-in

· a (reusable) message board to be displayed in the registration area (as close to the panel rooms as possible)

· reusable directional signs in the meeting site/building lobby indicating the location of the registration area

· name badges and table name tents (one color for review participants and a different color for all other attendees)

On-site Package:

The Contractor shall sort applications by panel, as assigned by the RA, and prepare registration packets for participants to be distributed at meeting check-in at the selected meeting site. The Contractor shall provide an on-site meeting package to include:

· Agenda to include the meeting site address and phone number, the telephone number for the Contractor, break times, and panel room location.

· Panel-specific order of the review of applications.

· Final Score Sheets for all applications in the assigned panel

· Reviewer Evaluation Forms

· Participant Roster for the assigned panel only, containing the names of the ORC panel members and their contact information.

· Chair package of one Short Set of the panel applications (application cover page, abstract, and other selected application parts, but never the entire application)

· One Short Set of the panel applications for each panel member other than the assigned applications

· One set of complete applications by panel

· Honoraria and reimbursement forms with instructions

· Logistics fact sheet, including list of local restaurants If the facility has a business center or similar facilities, the check-in table is to be located nearest the meeting rooms and used as an administrative center for printing of documents, staff coordination, and other Contractor administrative functions. If the facility does not have a business center or similar facilities, the Contractor can request approval from the PO for an on site office room for copying, printing, scanning, and email. HRSA will authorize the least expensive option of either an Internet connection or phone line for dial-up internet communications, but not both connections for the same meeting.

On the last day of the meeting, the applications and other confidential material are boxed for transport from the meeting site for shredding. The Contractor shall provide mailing labels and postage for the shipping and receipt of boxes as necessary.

Task 2(b) Review - Room Preparation.

Unless otherwise specified in the RFS, the orientation room shall be in theatre set-up, and the review shall be in U-shape with a head table and two small side tables for supplies & staff. Changes in room set-ups, requests for additional meeting space, or additional audiovisual requirements are handled through the Contractor. There shall be one set of basic office supplies for every review meeting. Unused and non-disposable supplies are recycled for subsequent meetings. The Contractor shall ensure that the rooms are ready before the ORC meeting. The Contractor shall place large boxes in all meeting rooms for discarding applications and other confidential review materials which shall be collected for confidential (shred) disposal by the Contractor staff at the end of the review.

Task 2(c) Review - Setting Up the Equipment.

HRSA will not furnish and/or transport appropriate computing and audiovisual equipment needed for each review. Contractor staff shall be familiar with the required equipment procedures for set up, operation, security, post-meeting disassembly and storage, and basic maintenance. All contract staff are responsible to insure the meeting room is equipped with review documents & forms, supplies and equipment before the start of the meeting.

For each meeting day, equipment shall be set up and tested in situ approximately one hour prior to the review to allow ample time to correct any malfunctions prior to the start of the meetings. The Contractor is also responsible for arranging appropriate security so that equipment and review materials are not left unattended in the meeting room during breaks and overnight between meeting days.

Typical equipment list for each panel:

· 1 laptop with Microsoft PowerPoint, WORD, Excel

· HRSA/DIR database templates loaded to hard drive

· 1 laptop power source cord

· Surge protector (s)

· A wireless print hub for printer economy

· Minimum of one Printer for up to 5 panels, with power source cord and cable to laptop

· LCD Projector with power source cord, remote control (one data, one summary), and 1 A-V cable to connect the lap top to the projector

· 2 three prong extension cord(s), 15 to 25 foot

· Additional laptops may be needed for selected meetings

· Extra batteries, cords, bulbs, printer cartridges, etc.

· Projection screen

Task 2(d) Review - Data Entry - The Federal RA will provide the Contractor with the Objective Review Summary Statement Template form. The Contractor shall provide staff to perform complete data recording and entry based on standard forms and procedures to be provided by the DIR, including, but not limited to:

Review participants’ scores (These score sheets are collected by Contractor staff from each of the review participants. Following the vote on the application and before consideration of the budget, the Contractor staff shall collect the individual score sheets from review participants and verify that each form is filled out completely and accurately.)

The Summary Statement Operator (SSO), with assistance from the RA, shall record the following information:

Attendance/ Number of votes to Approve or Not Recommend for Approval is recorded by the Chair (Absence for other than illness is not permitted. Recusal on grounds of a Conflict of Interest (COI) during the meeting must be immediately reported to the Federal RA and PO for immediate action.)(If Applicable)

Approval or “Not Recommend for Approval” (The Chair records the specific number of votes for Approval or Not Recommend for Approval of an application and the adhoc score in the database and on the Review Administrator’s summary score sheet.)(If Applicable)

Special Funding Factors (as noted in the Program Guidance):

· Preferences, if any (The Contractor staff inputs the assigned review participants’ determination of whether an applicant has requested and/or qualifies for a preference and the basis upon which the determination is made; e.g., some programs give preference to organizations that have specific capabilities such as telemedicine networking, or have established relationships with managed care organizations, noting the action taken with regard to any preferences and reasons why if not met but requested.)

· Priorities, if any (The Contractor staff inputs whether or not the priority, which causes a grant application to receive a fixed amount of extra rating points, was requested and/or met, noting the action taken with regard to any priorities and reasons if not met but requested.)

· Special Considerations, if any (The Contractor staff records whether the special consideration was requested and/or met, capturing reasons/rationale if not met.)

Recommendations (The contract Summary Statement Operator (SSO) inputs any formal communication from the review participants to applicants.)

Conditions of award, if any (The Contractor staff ensures that the condition(s) the Chair records is/are consistent with that captured during the discussion in the summary statement. If not, they check what was written on the Chair’s certification sheet and confer, through the Chair, with the assigned review participants until agreement on the text is reached among them.)

Review participants’ scores (The SSO shall capture on the consolidated Summary Statement the individual review participants scores and the final panel composite score.)

Budget figures and recommendation (Recorded on the individual score sheets and collected at the end of the application discussion.)

Summary Statement (The Summary Statement Operator (SSO) completes the text of the review participants’ consolidated summary statement and Contractor staff reviews it and makes any necessary changes to assure proper formatting, punctuation, and grammar, and eliminating any unnecessary code such as italics.)

During the ORC meeting, the Contractor staff shall maintain all documents, records and files in a separate application folder for each applicant. This folder always includes:

1. Abbreviated application to include:

a. face page,

b. abstract/summary,

c. Budget, budget narrative, and budget tables

2. Chair’s Certification of Review Results

3. Review participants’ original score sheets

4. Consolidated Final Summary Statement

Task 2(e) Review - Summary Statement.

The Contractor shall provide qualified staff to serve as the Summary Statement Operator, responsible for assuring that there is a consolidated final review summary statement produced during the ORC meeting for each application. The consolidated summary statement is the document sent to each applicant after the objective review, and summarizes the significant strengths and weaknesses of the applicant’s application as discussed during the review meeting. When voting is required by the program, it also indicates whether or not the application is recommended for approval. The Summary Statement shall also include the budget recommendations and funding factors.

To the extent possible, the Summary Statement Operator (SSO) shall copy the reviewer-prepared summaries for each grant review in a prescribed format to the laptop, or other electronic media, no less than 24 hours prior to the start of the ORC meeting. This is known as the draft consolidated Summary Statement. This draft Summary Statement should incorporate the comments of all three review participants, by section or criteria, with minimal modification. The form should be checked for grammar, punctuation, spelling, and contain all required review criteria or elements before being used as a template. The format should be proofed for correct HRSA announcement numbers, review dates, program criteria subheadings, and any other standard format information particular for that review.

The SSO shall operate a laptop computer linked with a projector and project the assigned reviewers summary reports and the draft consolidated Summary Statement onto the screen to be seen by the entire review panel for face-to-face and teleconferences. The SSO shall combine these review comments into one consolidated summary statement as directed by the Chair, until it becomes a record of panel consensus. The SSO shall verify the correct grant number and name information is in the summary document header and all sub headings.

The statement should not be considered a complete draft until it is read during the meeting for continuity, complete ideas and concepts, and comments clearly relevant to the program guidance and the application under review. The document must address the significant strengths and weaknesses of the application in clear and concise language as extracted with minimal modification from the reviewer drafts and discussed during the ORC discussion.

The consolidated summary statement is printed and placed in the application file, and the file is electronically saved in two locations (i.e., USB flash drive and laptop).

Task 2(f) Review - Debriefing.

At the conclusion of the review meeting, the Chair shall seek feedback from the review participants regarding aspects of the review, including continuity of the Program Guidance with the review worksheets, the review criteria and effectiveness of the review, effectiveness and efficiency of the review process, review process improvements for the future, etc. Contractor staff (SSO) shall record these debriefing data and, within 1 (one) business day, provide an electronic report to the Federal RA and the PO in a DIR specified format.

Task 2(g) Review - Evaluation Forms.

Reviewer evaluation forms included in the on-site materials shall be collected by the Contract staff and returned to DIR separated by panel within 14 (fourteen) business days. For Field and Teleconference meetings, the evaluation forms are mailed to the Contractor.

Task 2(h) Reconfirm Deliverable dates.

At the conclusion of a review meeting, the Federal RA and Contractor shall re-confirm all remaining critical dates and any adjustments for the post-review deliverables (e.g., Rank Order List [ROL], draft and final summaries, Official File). Changes shall be noted by the Contractor in an email to the Federal RA and the PO.

C. Post-Review

Post-Review Activities take place after the completion of a review. These activities include calculating overall scores; generating rank order lists (ROLs); submitting draft and final Summary Statements; transmitting review results; holding debriefings; completing the official review file; and compensating review participants.

Task 3(a) Post-Review – Submit Evaluation.

Within 5 days after each review meeting, the Contractor shall submit their individual evaluation of the review process with comments to the Project Officer (PO) in a format provided by DIR. The comments will be used as a Quality Assurance (QA) analysis for use in future meeting planning and operational issues of subsequent reviews. This is a separate task from the Contractor meeting with the PO for the quarterly contractor evaluation meeting.

Task 3(b) Post-Review - Disposal of Review Materials.

If the review is in a non-governmental facility, the Contractor staff shall pack boxes of all confidential materials to be shipped back to the Contractor for shredding following the reviews. If the review is held in a Government facility, HRSA reserves the option for on-site pick up by Government contract shredding.

All documents listed on the DIR Official Folder Checklist shall be scanned to the CD for archival at DIR. Once the electronic file content is verified to match the checklist by item and contain the data specific to the review participants and panels, the paper copies shall be shredded as confidential material.