Attachment 6 Technical Evaluation Criteria.pdf

RFP NN/LM 2011-2016

Attachment 6: Technical Evaluation Criteria

ATTACHMENT 6

TECHNICAL EVALUATION CRITERIA:

NATIONAL NETWORK OF LIBRARIES OF MEDICINE (NN/LM)

RFP NLM-10-055-AN

Criterion I: Mission, Goals, and Overall Plan (200 points)

A. Extent to which the proposal demonstrates: an understanding of the NN/LM program’s mission and goals; a sound assessment of the Region’s needs and priorities as they relate to the program goals and the information upon which that assessment is based; and the analysis of human and technological factors that may aid or inhibit progress toward reaching these goals.

B. Overall plan to develop, implement, and maintain a range of programs and services within the region which includes member libraries sharing resources to continue the infrastructure of the Network and active participation of Network members, including affiliate members, to carry out the Region’s outreach programs through subcontracts and other mechanisms.

C. The plan may include innovative approaches to providing services, which may include position sharing with another Region. The offeror may also propose programs in specific geographical areas and/or with specific groups, not excluding the provision of programs and services to all other areas of the Region or specific groups described in the Statement of Work. Where such programs are proposed, the offeror must demonstrate in-depth knowledge of the areas or groups through previous assessment and analysis, or an environmental scan.

D. Completeness in addressing all Statement of Work elements and other RFP requirements.

Criterion II: Technical Approach to Carrying out Network Infrastructure (200 points)

The logic, feasibility, and cost effectiveness of the approach to:

A. Develop and implement a program to provide health professionals in the Region with a basic level of information services; establish agreements with Network member libraries to provide services to unaffiliated health professionals; (20 points)

B. Assess and respond to the information needs of health sciences libraries to support programs that improve the transfer of health care and biomedical information within their institutions and through their participation in the Network; (20 points)

C. Manage the NN/LM Network membership program for the Region; (10

D. Implement the portion of the regional services plan concerned with document delivery to provide health professionals, including unaffiliated health professionals throughout the Region, with efficient, rapid access to health sciences information resources, and promote Network participation in Electronic Fund Transfer Service (EFTS). The plan shall conform to the NN/LM Resource Sharing Plan (Appendix 2); (15 points)

E. Enter into agreements with a limited number of institutions to serve as

Resource Libraries and negotiate with Resource Libraries that are willing to develop projects that will build on their expertise to address needs of the Region and to enhance Resource Libraries’ ability to contribute to their institutions goals and priorities; (20 points)

F. Implement the NN/LM National Emergency Preparedness & Response

Plan; serve as backup in an emergency to the Buddy RML; and serve as backup for NLM customer service inquiries; (10 points)

G. Identify and maintain effective methods of communication with current and potential NN/LM participants, with adherence to 508 compliance (http://www.section508.gov); (15 points)

H. Implement a Regional Advisory Committee (RAC). The RAC must include health professionals and health sciences librarians from hospital libraries and other large and small medical libraries within the Region. The RAC should also include representatives of state library agencies, public and school libraries, health information consumer groups, special populations, as well as representatives of community organizations with which the RML has partnerships. (10 points)

I. Monitor and evaluate the Region's programs to assess their effectiveness in meeting NN/LM goals and to identify and resolve problems, which impede the effective delivery of health information services; (15 points)

J. Participate in tests, usability studies, and user needs assessments of NLM and NN/LM products and services; (10 points)

K. Assist Network members with negotiating e-licensing agreements that meet the needs of their institutions and support the mission of the NN/LM by providing resources on and training in e-licensing issues; (10 points)

L. Provide consultation to Network members as well as community-based, faith-based and other organizations representative of special populations on preparing proposals for projects to compete for NLM and NN/LM funding; (10 points)

M. Promote and encourage the submission of applications for NLM-sponsored grants; (10 points)

N. Develop programs, which may include pilot projects, to assist the information professional in promoting evidence based health information in the institution. The programs shall identify health information issues and address the changing role of the information professional with a special emphasis on hospital librarians. The programs may include but not be limited to: advocacy, education and training to address knowledge management, clinical information systems, patient safety programs, electronic health records, health literacy, or patient education; (15 points)

O. Assist NLM in identifying collections of historical and unique materials related to the health sciences and increasing access to these materials (Appendix 3); (10 points)

Criterion III: Technical Approach to Carrying out Outreach Programs (300

The logic, feasibility, and cost effectiveness of the technical approach to:

A. Develop, implement, and evaluate outreach programs that bring biomedical information resources within easy reach of U.S. health professionals and the public health workforce, who are not directly affiliated with an institution with a medical library. The contractor shall focus on unaffiliated health professionals located in rural, inner city, and Medically Underserved Areas (MUA), public health workers, and minority health practitioners or those who serve minority populations. The outreach to health professionals program shall enlist the assistance of Network members through the use of subcontracts, agreements and other mechanisms. Specifically, the contractor shall undertake the following: (100 points)

1. Develop programs that reach practicing health professionals, including but not limited to: physicians, physician assistants, nurses, pharmacists, complementary and alternative medicine and allied health practitioners, veterinarians, public health workers, and health educators, with particular emphasis on collaborations with organizations/institutions that represent these groups; (25 points)

2. Develop programs that reach minority health practitioners and students in Historically Black Colleges and Universities, Hispanic Serving Institutions, Tribal Colleges, Community Colleges, and Vocational Schools, including institutions and organizations that focus on Native Hawaiians and Alaska Natives and the health professionals serving those populations or working in those disciplines. Programs should also focus on subject disciplines that have been identified as priority initiatives (e.g., health disparities, consumer health, health information literacy, HIV/AIDS, health services research, toxicology and environmental health); (25 points)

3. Develop pilot projects, which may include professional schools and organizations, to identify and promote the roles of libraries in institutions that have e-science initiatives; (20 points)

4. Develop programs, which may include pilot projects, with recently funded DHHS regional extension centers (RECs) (http://www.hhs.gov/news/press/2010pres/02/20100212a.html) to assist health professionals with implementing and using health information technology; (20 points)

5. Develop culturally and linguistically appropriate resources and provide training in the effective use of electronic health information resources.

Before developing any training materials, the MLA Educational Clearinghouse will be checked to avoid duplication. Copies of all training materials developed will be registered with the MLA Educational Clearinghouse. Training should include, but not be limited to: PubMed, MedlinePlus, and other NLM resources. (10 points)

B. Develop, implement, and evaluate outreach programs to increase the public’s awareness of and access to high quality electronic health information. The contractor shall work with a variety of intermediaries including Network members, health professionals, public health workers, information professionals, educators, community, faith-based, volunteer, and other types of organizations, including those serving minorities, special and underserved populations, health advocacy and self-help groups, through the use of subcontracts, agreements, and other mechanisms: (130 points)

1. Develop programs that reach special populations (e.g. minorities, seniors, teens, veterans’ groups, low income populations, etc.), focus on subject disciplines (e.g. health disparities, health information literacy, HIV/AIDS, public health, culturally and linguistically appropriate information, personal health record, emergency preparedness, etc.), and promote NLM resources to consumers; (40 points)

2. Foster collaborations and encourage outreach partnerships among Network members, community, faith-based, and volunteer organizations, and other types of organizations, to improve access to electronic consumer health information at the local, state, and regional levels; (35 points)

3. Provide consultation for Network members in project planning, preparation, writing and evaluation; (10 points)

4. Develop culturally and linguistically appropriate resources and provide training in the effective use of electronic health information resources. Before developing any training materials, the NLM Training Center and MLA Continuing Education Clearinghouse will be checked to avoid duplication. Copies of all training materials developed will be registered with the MLA Continuing Education

Clearinghouse. Training should include, but not be limited to:

PubMed, MedlinePlus, and other NLM resources; (15 points)

5. Develop pilot projects to identify and promote the roles of libraries in institutions that have received or are seeking NIH Clinical and Translational Science Awards (CTSA) with a focus on community engagement. (30 points)

C. Promote NLM and NN/LM programs and services at national, regional and state meetings of health professionals and organizations that represent consumers: (30 points)

1. Approach and strategy for identifying a minimum of eight meetings (four national and four regional, state, or local) for an exhibit, program presentation, or course offering, including making all arrangements (scheduling, fees, shipping, logistics, equipment and publications), and designating an RML staff member who is responsible for identifying health professional and consumer-focused meetings, reporting exhibits and other RML activities at all national, regional, state and local meetings, in consultation with the National Network Office; (10 points)

2. Attend state library association meetings and exhibit or make presentations to promote NLM and NN/LM programs and services; (10

3. Enlist the assistance of Network members in exhibiting at national, regional, state, and local meetings. (10 points)

D. Develop an information technology and policy awareness program which includes: Regional forums that focus on e-science initiatives related to the use of information technology to improve access to biomedical information and on policy issues affecting production, distribution, and access to information and monitoring current technological and policy developments and trends to identify those that will improve access to biomedical information; (25 points)

E. Develop and implement an NN/LM technology improvement program, if needed, to encourage high quality health information access and delivery to Network members, health professionals and consumers in the Region.

Emphasis should be on the “underconnected,” particularly in inner city and rural health organizations, to upgrade and maintain technology access. If no program is proposed, the contractor shall provide an explanation of why the program is not needed. Site selection requirements are found in (Appendix 5); (20 points)

Criterion IV: Personnel (175 points)

A. Demonstrated ability of the individual proposed as RML Director to provide the leadership required for a program of this scope. Evidence that the individual meets the following requirements: (40 points)

1. Progressively responsible management experience in large health sciences libraries as well as progressively responsible experience in cooperative program planning, development, and implementation;

2. Demonstrated ability to develop and implement outreach programs.

B. Demonstrated ability of the individual proposed to manage the RML staff and administer day-to-day operations to meet the following requirements: (70

1. Progressively responsible management experience in large health sciences libraries; experience in the coordination, design, and successful implementation of outreach programs for health professionals and consumers;

2. Experience with grants/contracts; experience in preparation of budgets;

3. Demonstrated writing skills and experience in interpersonal and interorganizational relations;

4. Understanding of basic public health policies and national health objectives.

C. Evidence of expertise for proposed staff in the areas of responsibility for each position, If individuals have not been selected, position descriptions and a list of qualifications for each position should be submitted for evaluation; (50

D. An organizational chart showing the position of the library within the institution, as well as an organizational chart showing various positions in the RML office, their functions, and the reporting structure; (10 points)

Criterion V: Facilities, Resources, and Institutional Commitment (125 points)

A. The proposal will be evaluated upon the strength of the statement of the willingness, commitment, and ongoing support of administrators associated with the parent institution to the goals and programs of the NN/LM; (50 points)

B. The proposal will also be evaluated on: (75 points)

1. The evidence that the organization possesses physical facilities adequate to support the NN/LM program; description of training facility for use by the NLM Training Center and the RML for regional training needs;

2. The organization's prior demonstrated commitment to providing information outreach services for health professionals and general public; the organization's demonstrated expertise in information and computer technology, including the national information infrastructure;

3. The organization's active participation in NN/LM programs in the

Region during the past five years (2006-2011);

4. The resources and unique services of the organization which enhance its ability to perform as an RML.

TOTAL WEIGHT: 1,000 points

NATIONAL LIBRARY OF MEDICINE TRAINING CENTER

I. Understanding of the requirements and Offeror's technical approach

A. Comprehension of the purpose and objectives of the NLM Training Center;

understanding of the technical requirements of the NLM Training Center. (20

B. Description of the management and coordination required for the NLM

Training Center, including the logic, feasibility, and cost effectiveness of the approach to: (35 points)

1. Assuring that two instructors are available to travel to each training sites to share training responsibilities and that adequate back-up is available in case of unavailability of one or both of the instructors;

(5 points)

2. Developing the training schedule, including selecting the training sites, and determining equitable geographic distribution based upon demand for training; (5 points)

3. Maintaining and updating NLM Training Center Web site and continuing development of the MLA Continuing Education Clearinghouse database, including usability testing; (5 points)

4. Coordinating and providing instructional design expertise for

NN/LM’s development, review and maintenance of Web-based tutorials and adherence to 508 compliance (http://www.section508.gov); (5 points)

5. Planning and delivering training through desktop webcasts; (5

6. Conducting evaluations/post-training assessment of training participants; (5 points)

7. Publicizing classes and developing and maintaining the registration system for classes, including NCBI and UMLS classes. (5 points)

SUBTOTAL 55 points

II. Personnel

Proposed personnel will be evaluated upon their background and experience which is pertinent to the tasks described in the Statement of Work, specifically:

A. Demonstrated technical expertise with the NLM Web-based search interfaces; (7 points)

B. Demonstrated ability to plan and administer a program of this type; at least one year of experience in organizing and planning training sessions or workshops; a thorough knowledge of NLM web-based search interfaces and related products and services, and the NN/LM Training Center and Clearinghouse and its objectives; (7 points)

C. Demonstrated instructional design expertise and other experience related to the development, review and maintenance of Web-based tutorials; (7 points)

D. Demonstrated expertise and experience related to identifying topics for desktop webcasts and other Web-based broadcasts for training as well as testing and archiving webcasts; (7 points)

E. Demonstrated ability, training and/or experience in adult education and public speaking. (7 points)

SUBTOTAL 35 points

III. Institutional Experience and Commitment

Demonstrated institutional experience in conducting similar work; evidence of institutional resources and the support of institutional administrators adequate to support the proposed program. (10 points)

SUBTOTAL 10 points

TOTAL POINTS POSSIBLE 100 points

NATIONAL NETWORK OF LIBRARIES OF MEDICINE

OUTREACH EVALUATION RESOURCE CENTER

A. Comprehension of the purpose and objectives of the NN/LM Outreach Evaluation Resource Center; understanding of the technical requirements of the NN/LM Outreach Evaluation Resource Center; (15 points)

B. Description of the management and coordination required for the NN/LM

Outreach Evaluation Resource Center, including the logic, feasibility, and cost effectiveness of the approach to: (25 points)

1. Provide or arrange for training and continuing education opportunities for NLM, RML and Network staff with regard to planning and evaluation of outreach and basic Network services and maintain adherence to Section 508 compliance (http://www.section508.gov); (3 points)

2. Assist and coordinate the evaluation-related activities and facilitate communication with the RML staff identified as regional evaluation consultants; (3 points)

3. Communicate results of consultations and encourage sharing among Network members and RML staff; (3 points)

4. Facilitate the identification, promotion and dissemination of best practices and lessons learned via the outreach applications, online reporting system and other mechanisms; (3 points)

5. Consult with Network members on specific outreach evaluation projects; (3 points)

6. Assist on a selective basis in the analysis, synthesis, and reporting of evaluation results, including those of cross-regional efforts; (3 points)

7. Revise the guides on outreach planning and evaluation to include concepts related to collaboration with community-based organizations, participatory evaluation and program logic models; (2.5 points)

8. Assist with the implementation of concepts from the guides on outreach planning and evaluation; (2.5 points)

9. Assist as needed with the evaluation efforts undertaken by the NLM Training Center and Web-STOC; (1 point)

10. Participate in a review of the Center, to be conducted at least once during the contract, by selected representatives from NLM and the RMLs. (1 point)

SUBTOTAL 40 points

A. Demonstrated ability to plan and administer an evaluation program, including expertise in evaluation methodology, consultations, curriculum development and training; demonstrated ability in the analysis, synthesis, and the reporting of evaluation results, best practices and lessons learned; (20 points)

B. Demonstrated familiarity with the nature and scope of the NN/LM's past evaluation efforts; demonstrated technical expertise with outreach evaluation; (15

C. Demonstrated ability to implement multiple communications methods for information dissemination on evaluation to RML staff and Network members;

demonstrated ability in development of Web-based guides. (10 points)

SUBTOTAL 45 points support the proposed program. (15 points)

SUBTOTAL 15 points

TOTAL POINTS POSSIBLE 100 points

NATIONAL NETWORK OF LIBRARIES OF MEDICINE

WEB SERVICES, TECHNOLOGY & OPERATIONS CENTER

A. Comprehension of the purpose and objectives of the NN/LM Web Services, Technology & Operations Center; understanding of the technical requirements of the Center; (15 points)

B. Description of the management and coordination required for the Center, including the logic, feasibility, and cost effectiveness of the approach to: (35

1. Develop and maintain reliable Web services for NN/LM’s public and internal needs. This entails critically assessing and continuously improving the technical architecture of NN/LM’s Web services, including the archiving of all NN/LM data and systems architecture and implementation of a Content Management System; (20 points)

2. Ability to collaborate with the national centers, regional staff, National Network Office, and advisory groups, incorporating feedback for effective operations; (4 points)

3. Establish and maintain channels of communication among Regional

Medical Library (RML) staff, Network members, and the National Network Office regarding critical issues and updates affecting NN/LM’s Web services and adherence to 508 compliance (http://www.section508.gov);

(4 points)

4. Provide support to NLM, NN/LM, and the national centers, for various systems including DOCLINE, outreach applications, online reporting system, the MLA Continuing Education Clearinghouse, and identification of appropriate software and emerging technologies; (4 points)

5. Assist NLM, the RMLs and other NN/LM centers with the implementation of the NN/LM and NLM’s emergency preparedness plans; (2 points)

6. Participate in a review of the center, to be conducted at least once during the contract by selected representatives from NLM and the RMLs. (1 point)

SUBTOTAL 50 points

A. Demonstrated ability to plan and administer a Web and technology services program; (10 points)

B. Demonstrated ability to develop, install and maintain Web-based applications, Web servers, Web software development utilities, database drivers and related systems software (25 points);

C. Demonstrated ability to summarize and communicate technical subjects effectively for non-technical staff. (5 points)

SUBTOTAL 40 points support the proposed program. (10 points)

SUBTOTAL 10 points

TOTAL POINTS POSSIBLE: 100