B08_ATT_6_PEER_REVIEW_GUIDANCE_DOCMENT.pdf

NPS Interim Peer Review, Information Quality Correction Guide and Ethics Statement--January 31, 2008

NATIONAL PARK SERVICE 12

U.S. DEPARTMENT OF THE INTERIOR 14

Interim Guidance Document Governing Code of Conduct, Peer 17

Review, and Information Quality Correction for National Park 18

Service Cultural and Natural Resource Disciplines 19

/s/ Herbert C. Frost 33 Herbert C. Frost 34 Acting Associate Director 35 Natural Resource Stewardship and Science 36 January 31, 2008 38 This guidance document will remain in effect until amended or superseded. 41

Interim Guidance Document Governing Code of Conduct, Peer 1 Review, and Information Quality Correction for National Park Service 2

Cultural and Natural Resource Disciplines 3

Contents: 4

I. Background and Purpose 5 II. Authority to Issue this Reference Manual 6 III. Code of Scientific and Scholarly Conduct 7 IV. Information Quality 8 V. Thresholds for Peer Review 9 VI. Peer Review Policies and Instructions 10 VII. Responsibilities 11 VIII. Definitions 12 IX. Legal Effect 13

__________________________________________ 14

I. BACKGROUND AND PURPOSE 16

I.a. Background 17 Sound management and interpretation of National Park System resources and National Park Service 18 technical assistance programs depend on authoritative information from scientific and scholarly 19 activities. Peer review provides the appropriate process for ensuring technical quality and accuracy 20 of these activities, which include inventory, monitoring, research, assessment, and management 21 projects. This peer review must be applied appropriately at all stages, which may include proposals 22 at the funding decision point, detailed proposals or plans of action at the point of initiating an 23 activity, progress of an activity at key points during its duration, and results and draft reports of 24 accomplishments of the activity. 25 The scientific and scholarly guidance presented in this document ensures that the review 27 requirement is met using a consistent, conscientious, and appropriate level of effort. Such technical 28 peer review is essential to demonstrate the professional stature and ensure the accountability of the 29 National Park Service's acquisition and application of scientific and scholarly information. Such 30 scientific peer review complements, and is part of, administrative review. 31 I.b. Purpose 33 NPS issues this guidance to ensure that NPS scientific and scholarly activities comply with the 34 OMB Final Information Quality Bulletin for Peer Review (70 FR 2664-2677), the Department of 35 the Interior draft 305 DM 4 Peer Review, NPS Director’s Order 11B: Ensuring Quality of 36 Information Disseminated by the National Park Service, and the Secretary of the Interior’s 37 December 21, 2007 memorandum “Ensuring Integrity in Scientific Activities”. This guidance is 38 expected to: 39

A. Enhance the quality, objectivity, utility, and integrity of scientific and scholarly 41 information generated or supported by the NPS; 42

B. Increase the credibility of decisions to which these and other scientific information 1 contributes; and 2 C. Extend application of the concepts of the OMB Final Information Quality Bulletin for 4 Peer Review (70 FR 2664-2677), henceforth referred to as the OMB Peer Review Bulletin, to 5 information disseminated by the Service that is neither a highly influential scientific 6 assessment nor influential scientific information. 7

I.c. Policy 9 The NPS shall ensure appropriate peer review of all scientific and scholarly information prior to 10 use in decision-making, regulatory processes, or dissemination to the public and regardless of 11 media (i.e., print, digital, audiovisual, or Web). Any information that the NPS determines to be 12 either “influential scientific information” and/or “highly influential scientific assessment” will 13 require more specific, independent peer review consistent with the OMB Peer Review Bulletin. 14 NPS will place these guidelines, its peer review agendas for influential scientific information and 15 highly influential scientific assessment (as defined by the OMB Peer Review Bulletin), and its 16 information quality correction site on a publicly available webpage. These guidelines become 17 effective on an interim basis February 1, 2008. 18 I.d. Scope 20 These guidelines apply to all scientific and scholarly information and assessments produced, 21 used, or sponsored by the NPS. The guidelines identify the ethical standards within which 22 employees and volunteers will conduct NPS-sponsored scientific and scholarly activities. The 23 peer review component of these guidelines applies to scientific and scholarly activities. It does 24 not apply to policy or management decisions, although it does apply to the underlying scientific 25 and scholarly information that, along with other factors, informs a decision maker. For example, 26 proposed management options associated with Environmental Impact Statements (EIS) or other 27 parts of the NEPA process would not be subject to peer review whereas consolidated technical 28 components of an EIS that present a scientific or scholarly evaluation or other related products or 29 analytical models are subject to peer review. These guidelines do require that all scientific and 30 scholarly information disseminated to the public in any format meets the requirements of D.O. 31 11B: Ensuring Quality of Information Disseminated by the National Park Service. 32 I.e. What Is “Peer Review”? 34 Peer review is the critical evaluation of the scientific or scholarly merits of an activity conducted by 35 impartial subject-matter experts who are not directly associated with the activity. Peer review may 36 be formal or informal. It may be authored or anonymous. It may involve open exchange of views 37 and ideas between reviewers and managers of the activities being reviewed during the process of the 38 review, or it may consist of reviewers conducting their observations and writing their reports 39 without any contact with those managers. No matter how it is conducted, it must be objective to 40 achieve its purpose of ensuring that the quality, objectivity, utility, and integrity of scientific and 41 scholarly information meets scientific, scholarly, and technical community standards. 42 Peer review may be conducted by scientists and scholars from the National Park Service, other 44 federal or state agencies, universities, or other organizations. Peer reviewers may submit their 45 comments separately to the National Park Service or they may submit their comments jointly as part 1 of a team report, subject to appropriate provisions of the Federal Advisory Committee Act. 2 Peer review is different from public comment and other stakeholder processes, which are not 4 addressed in this document. The selection of peer reviewers is based on their demonstrated 5 expertise, independence, and lack of conflict of interest. 6 I.f. What is Administrative Review? 8 The scientific and scholarly peer review process outlined here does not eliminate the need for 9 administrative review of scientific and scholarly activities by managers. Managers conduct 10 administrative review to ensure that: 11

• proposed activities are compatible with Park Service policies and regulations; 12

• proposed activities are pertinent to park purposes, programs, and needs; and 13

• personnel, logistical (e.g., housing, permits, and transportation), and supplementary (e.g., 14 equipment and supplies) operating needs are identified and authorized prior to commencement 15 of a project. 16

II. AUTHORITY TO ISSUE THIS REFERENCE MANUAL 18

II.a. Guidance for Science, Peer Review, and Information Quality 19

305 DM 2 requires that science must be integrated into and used in setting regulatory and 20 management policies in the Department and its bureaus; 21

DOI Information Quality Guidelines establish policy to ensure and maximize the 23 objectivity, utility, and integrity of information disseminated to the public by the 24 Department. These guidelines implement the Information Quality Act (P.L. 106-554 25 Section 515) and associated OMB Guidelines (67 FR 8452-8460); 26

E.O. 12866 establishes a government-wide policy that each agency shall base its regulatory 28 decisions on the best reasonably obtainable scientific, technical, economic and other 29 information; and 30

OMB Final Information Quality Bulletin for Peer Review (70 FR 2664-2677) establishes 32 government-wide requirements for the peer review of “influential scientific information” 33 and “highly influential scientific assessments.” It also establishes requirements for posting 34 peer review plans on a website and annual reporting of associated peer review activities to 35 the OMB. 36

II.b. Guidance for Scientific Code of Conduct 38

5 U.S.C. 301 allows the head of an executive department to prescribe regulations for the 39 conduct of its employees; 40

43 CFR 20.501 requires employees of the Department to comply with all Federal statutes; 42

Executive Orders; and Office of Government Ethics, Office of Personnel Management, 43 and Departmental regulations; 44

43 CFR 20.502 states that employees are required to carry out the announced policies and 1 programs of the Department; 2

43 CFR 20.502(a) states that an employee is subject to appropriate disciplinary action if he 4 or she fails to comply with any lawful regulations, orders, or policies; 5 Federal Policy on Research Misconduct, 65 FR 76260-76264, December 6, 2000; and 7 Standards of Ethical Conduct for Employees of the Executive Branch, 5 CFR 2635. 9

II.c. General Authority 11

16 U.S.C. 1 through 4 (the National Park Service Organic Act). 12

II.d. Other Relevant Policy and Guidance 14 NPS Management Policies 2006 15

Chapter 1, The Foundation, Sections 1.8 (Managing Excellence); 1.9.4 (Public 16 Information and Media Relations); 17 Chapter 2, Park System Planning, Sections 2.1.2 (Scientific, Technical, and Scholarly 19 Analysis), 2.1.3 (Public Participation); 2.3.1.4 (Science and Scholarship); 20 Chapter 4, Natural Resource Management, Sections 4.1.2 (Natural Resource 22 Information); 4.2 (Studies and Collection); 4.8.2.1 (Paleontological Resources and Their 23 Contexts); 4.8.2.2 (Caves); 24 Chapter 5, Cultural Resource Management, Sections 5.1.1 (National Park Service 26 Research); 5.2.3 (Confidentiality); 5.3.5.3.2 (Sacred Sites); 5.3.5.5.4 (Acquisition, 27 Management, Disposition, and Use); 5.3.5.5.6 (Archives and Manuscripts); 28 Chapter 7, Requirements for All Interpretive and Educational Services, Section 7.5.4 30 (Research and Scholarship); 31 Chapter 8, Use of the Parks, Sections 8.5 (Use by American Indians and Other 33 Traditionally Associated Groups); 8.11 (Social Science Studies); 34 Chapter 10, Commercial Visitor Services, Section 10.2.4.9 (Natural and Cultural 36 Resource Management Requirements). 37

DO #11B Ensuring Quality of Information Disseminated by the National Park Service; 39 DO #12 Environmental Impact Analysis; 41 DO #19 Records Management; and 43 DO #78 Social Science, Section III.I (Peer Review). 45

III. CODE OF SCIENTIFIC AND SCHOLARLY CONDUCT 1

To enhance their contribution to quality, objectivity, utility, and integrity of such information, all 2 NPS employees working with scientific and scholarly information will, in performing their 3 duties: 4 act in the interest of the advancement of knowledge and contribute the best, highest 5 quality scientific and scholarly information for the NPS; 6 conduct, process data from, and communicate the results of scientific and scholarly 7 activities honestly, objectively, thoroughly, and expeditiously; 8 be responsible for the entrusted resources, including equipment, funds, work time, 9 employee work time, and prompt and accurate use and reporting of financial resources 10 and scientific and scholarly work; 11 fully disclose all research methods used, available data, and final reports and publications 12 in a timely manner and consistent with applicable laws and policy; 13 respect, to the fullest extent permitted by law, confidential and proprietary information 14 regarding interests and resources that are studied or affected by scientific or scholarly 15 activities or the resulting information; 16 neither hinder the scientific or scholarly activities of others nor engage in dishonesty, 17 fraud, deceit, misrepresentation, coercive manipulation, or other scientific or scholarly 18 misconduct; 19 welcome constructive criticism of scientific and scholarly activities, welcome and 20 participate in appropriate peer reviews, critique others’ work respectfully and objectively, 21 and substantiate comments with care; 22 be diligent in creating, using, preserving, documenting, and maintaining collections and 23 data, ensuring established quality assurance and quality control programs, follow the 24 NPS’s records retention policies, and comply with Federal law and agreements related to 25 use, security, and release of confidential and proprietary data; 26 adhere to appropriate standards for reporting the results of scientific and scholarly 27 activities, including respecting the intellectual property rights of others; 28 to the extent possible and practical, differentiate among facts, opinions, hypotheses, and 29 professional judgment in reporting the results of scientific and scholarly activities to 30 others, including scientists, decision makers, and the public; 31 be responsible for the quality of collected data and interpretations, and for the integrity of 32 conclusions drawn in the course of scientific and scholarly activities; and 33 place integrity, utility, and objectivity of scientific and scholarly activities and reporting 34 of their results ahead of personal gain or allegiance to individuals or organizations. 35

IV. INFORMATION QUALITY 37

IV.a. Administrative Record 38 Whenever the NPS relies on influential scientific information (including a highly influential 39 scientific assessment) to support a regulatory action or policy decision, it shall include in the 40 administrative record for that action a certification explaining how the NPS has complied with 41 the requirements of the OMB Peer Review Bulletin and the Information Quality Act. 42 IV.b. Information Quality Correction Procedures 44 IV.b.1. Complaints About Quality of Scientific and Scholarly Information 45

Persons who have complaints about NPS-provided scientific and scholarly information may avail 1 themselves of any of the four methods outlined in D.O. 11B for notifying the NPS of their 2 complaints. These four methods include informally discussing their complaint in person with 3 park or program office staff, informally directing complaints about the quality of disseminated 4 information to the superintendent of the park or manager of the program office responsible for 5 disseminating the information, formally directing complaints about the quality of disseminated 6 information by mail to the NPS Washington Administrative Program Center, or formally 7 directing complaints to the Correspondence Control Unit by e-mail. 8 Persons submitting a formal request for correction of such information must submit all 10 information identified in D.O. 11B, the technical components of which include the specific 11 reasons for believing the information fails to meet OMB, DOI, or NPS standards, along with any 12 supporting documentation; a detailed description of the specific material in question, including 13 where the material is located (that is, publication title, date, and publication number, if any; the 14 website and web page address; or other source where the material exists); and the specific 15 recommendations for corrective action. 16

Upon receipt of a formal request for correction of information, NPS shall post the request on the 17 appropriate park or program office Information Quality web page with a link to the NPS web 18 page (www.nps.gov/notices.htm). All interim and final responses also shall be posted on the 19 appropriate web page at the time they are sent to the complainant. 20

IV.b.2. Processing Complaints 21 The CCU will route formal complaints it receives regarding scientific or scholarly information to 22 the park or office that disseminated the information and track response to assure that the 23 response complies with the requirements of Director's Order 11B. The park or office receiving 24 the complaint, regardless of the manner of receipt, will notify the complainant of receipt within 25 10 working days. The disseminating office will evaluate the complaint within 60 calendar days 26 of the day it is received by the NPS, in accordance with the OMB guidelines, and notify the 27 complainant as to whether the information has been corrected, deleted, or confirmed to be 28 accurate. The Service will respond to additional complaints on the same subject according to the 29 processes delineated in D.O. 11B. 30

IV.b.3. Appeals Process 31

If a complainant does not receive the notice or the response within the time frames described 32 above, or wishes to appeal a determination of merit, or wishes to appeal the proposed correction 33 of information, the complainant may appeal to the Director, National Park Service. Appeals must 34 follow the requirements in D.O. 11B. All appeal requests shall be posted on the park or program 35 office Information Quality web page upon receipt, with a link to the NPS Information Quality 36 web page. 37

If the Director determines that an appeal of a determination has merit or the proposed correction 38 of information has merit, the affected program office or park will be notified. The challenged 39 information will be withdrawn, to the extent practicable, from the public domain and will not be 40 used in any NPS decision-making process until it is corrected. The Director will make a decision 41 on the final appeal within 60 calendar days. 42

V. THRESHHOLDS FOR PEER REVIEW 2

V.a. When is Peer Review Required? 3 Although peer review generally benefits all scientific and scholarly activities, existing workloads do 4 not permit conducting the same intensity of peer review in every case. As a result, supervisors 5 apply professional judgment to each activity to determine what level of peer review is required at 6 any step in the implementation process. In general the need for peer review should be decided by a 7 supervisor or program leader at least one level above the person who conducts the work. The 8 following factors are examples of considerations used in making this professional judgment: 9 the amount of funding involved; 11 the number of researchers involved; 12 the length of time involved; 13 whether or not the activity involves significant experimentation or use of new technology or 14 methods; 15 the degree to which the information may contradict prior findings and results or is likely 16 to be precedent-setting; 17 potential for the activity to have applicability to other similar projects; 18 the geographical scale involved; 19 specific properties of, and values assigned to, natural and cultural resources involved; 20 the potential for risk to both target and non-target resources; 21 the potential for risk to visitor enjoyment or park operation; 22 whether the act of conducting the activity or the probable results of the activity have 23 potential to generate broad implications beyond the project, itself; 24 the probable degree of controversy likely to be associated with the activity or the 25 management, regulatory, or policy decision that it informs; 26 the potential for societal and resource impacts associated with management, regulatory, or 27 policy decisions that the information might influence; and/or 28 the presence of legal mandates or unusual, unanticipated, or time-sensitive requirements 29 that might preclude carrying out peer review. 30 V.b. Scientific and Scholarly Information Not Requiring Peer Review. 32 Activities that generally do not require independent peer review are those that are unlikely to be 33 controversial or of a sensitive nature; involve routine, well-established practices; and pose no 34 unacceptable impact to resources, visitor enjoyment, or operations. In such cases, administrative, 35 NEPA, and Section 106 review may provide a sufficient alternative to peer review. When in doubt, 36 however, it is better to conduct peer reviews. 37 Other information that does not require peer review under this guidance includes: 39

A. Items found in the list of exemptions under Section IX of the OMB Peer Review Bulletin: 40 related to certain national security, foreign affairs, or negotiations involving international 41 trade or treaties where compliance with this Bulletin would interfere with the need for 42 secrecy or promptness; 43 disseminated in the course of litigation, adjudication or a permit proceeding (including a 44 registration, approval, licensing, or a site-specific determination), unless the agency 45 determines that peer review is practical and appropriate and that the influential 46 dissemination is scientifically or technically novel or likely to have precedent-setting 1 influence on future adjudications and/or permit proceedings; 2 a health or safety dissemination where the agency determines that the dissemination is 3 time-sensitive (e.g., findings based primarily on data from a recent clinical trial that was 4 adequately peer reviewed before the trial began); 5 an agency regulatory impact analysis or regulatory flexibility analysis subject to 6 interagency review under Executive Order 12866, except for underlying data and 7 analytical models used; 8 routine statistical information released by federal statistical agencies (e.g., periodic 9 demographic and economic statistics) and analyses of these data to compute standard 10 indicators and trends (e.g., unemployment and poverty rates); 11 accounting, budget, actuarial, and financial information, including that which is generated 12 or used by agencies that focus on interest rates, banking, currency, securities, 13 commodities, futures, or taxes; or 14 information disseminated in connection with routine rules that materially alter 15 entitlements, grants, user fees, or loan programs, or the rights and obligations of 16 recipients thereof. 17

B. Information published either in a peer-reviewed journal listed on the Thomson Scientific 19 Master Journal List or in an official report of the National Academies of Sciences. 20 C. Routine statistical data used to compute standard indicators and trends that are gathered 22 using methods based on well-established, peer-reviewed protocols and are analyzed and 23 interpreted within the guidelines of the protocols. 24 D. An Environmental Impact Statement prepared as part of the NEPA process, except for 26 underlying scientific and scholarly products and analytical models that provide information 27 serving as the principal basis for a decision. In such cases, peer review that was conducted 28 must have been appropriate to the context in which the information will be used or an 29 additional peer review will be required. 30 E. Information performed by other federal agencies following the requirements of the OMB 32 Peer Review Bulletin. 33

VI. PEER REVIEW POLICIES AND INSTRUCTIONS 35

VI.a. The Peer Review Process 36 The peer review process begins when the project manager (or supervisor) of a scientific or scholarly 37 information activity, such as a proposal, nomination, progress report, review of previous studies, or 38 product, identifies the need for peer review. Project managers or supervisors may be program 39 scientists or scholars, resource managers, science coordinators, or other professionals. 40 The submission, which may be in the form of a proposal, study plan, inventory and monitoring plan, 42 conceptual model, protocol, interim report, draft report, database, action plan, or scientific 43 assessment should be peer reviewed in accordance with the following peer review process prior to 1 funding, authorization, execution, publication, or incorporation into a decision: 2

A. The project manager assigns the submission to a peer review manager, who is not the 4 performer of the activity but may be the project manager, to provide appropriate technical 5 oversight and ensure appropriate quality performance throughout the life of the activity. The 6 project manager consults appropriately with park, regional office, and Washington office 7 experts to obtain advice on selecting the best-qualified peer review manager. For some 8 activities, a park resource management specialist may be the most appropriate peer review 9 manager. For other activities, the peer review manager might come from other NPS offices 10 at the regional, network, or national level. The peer review manager ensures that an 11 appropriate critical review process is conducted of the activity, determines validity of 12 reviewer comments received, ensures that valid comments are incorporated into the activity, 13 and ensures that records of the reviews, revisions, and approvals are properly filed and 14 archived. The peer review manager may also serve as the contracting/agreement officer's 15 technical representative. 16

B. The peer review manager identifies the structure of the peer review and recruits one or more 18 highly qualified individuals with technical expertise in the specific appropriate disciplines to 19 provide peer review of each relevant component of the activity. Peer reviewers may be from 20 within or outside the Park Service. Peer reviewers may have neither direct involvement with 21 the activity nor a direct stake in the outcome of the review. The peer review manager may 22 obtain recommendations for appropriately skilled reviewers from other knowledgeable 23 people, in addition to using competent reviewers known to the peer review manager. 24 Depending on the nature of the peer review, the peer review manager may serve as one peer 25 reviewer if otherwise qualified. The reviewers carry out their tasks independently, without 26 interaction among themselves, or together, depending on the structure of the review process. 27

The peer review manager may determine that an alternative peer review process may be 29 accepted whenever it can be demonstrated that a reputable scientific body has already 30 completed a formal peer review on a scientific or scholarly activity or product. (See Section 31 IV.b. Scientific and Scholarly Information Not Requiring Peer Review.) Documentation 32 of reliance on this alternative review process should be included in the peer review file. 33

C. The peer review manager ensures that peer review adequately addresses the technical merits 36 of the methods, the professional techniques, data analysis, conclusions, and suggested 37 management outcome scenarios and that activities requiring significant data analysis or 38 sampling design receive additional review by peer reviewers with expertise in statistics and 39 experimental design. 40

D. Because reviews of funding proposals must be performed in a timely manner, given the 43 necessity to obligate funds within a given fiscal year and the short time usually available for 44 obligating funds the peer review manager should schedule peer reviews to allow enough 45 time for the development of cooperative agreements or contracts, including solicitor review. 46

Therefore, it is critical that reviewers be selected who can provide fast turnaround as well as 1 scientifically sound reviews. 2

E. Proposals solicited through competitive procurement or cooperative agreement processes will 4 be subjected to a technical review process selected by the contracting/agreement officer 5 upon recommendation of the contracting/agreement officer's technical representative. 6 Typically, the peer review panel and the technical review panel are one and the same. 7

F. The peer review manager will negotiate with the performer of the activity regarding revisions 9 to the information as recommended by peer reviews, will ensure that the project manager is 10 notified of these recommended revisions, and will maintain a record of the reviews, activity 11 revisions, and approvals in accordance with NPS records disposition requirements. 12

G. Once all appropriate changes and modifications have been completed in response to peer 14 review comments, the peer review manager recommends technical acceptance of the activity 15 in a written record documenting the process. The project manager has the final authority to 16 either approve or reject an activity. 17

In those cases where peer reviewers recommend rejection of the activity, the peer review 19 manager should write a summary of the reasons for the recommendation and be prepared to 20 supply the author with reviewer comments. 21

VI.b. Peer Review Objectives 24 The peer review manager should define clearly the objectives of each peer review using 25 questions that all reviewers are given and are expected to answer. Examples of potential peer 26 review questions include: 27

- Are purpose for and methodology underlying the activity clearly presented, appropriate, 28 and valid? 29

- Are appropriate field, laboratory, library, and statistical methods used for data collection, 30 analysis, and characterization of uncertainty, and are they sufficiently described to allow 31 for replication? 32

- Does the communication plan address communication needs and audiences 33 appropriately? 34

- Is the combination of involved scientific, scholarly, or technical disciplines sufficient to 35 adequately measure and test the hypothesis or to meet the activity’s stated objectives? 36

- Is the work original? 37

- Does the activity address significant, new, or novel circumstances? 38

- Does the activity include an appropriate literature review that puts the work in the context 39 of previous work? 40

- Are conclusions reasonable and supported by data and other evidence and are alternative 41 conclusions or counter arguments taken into account? 42

- Is written information well-written, logical, and understandable? 43

- What are the strengths, limitations, and potential usefulness to park managers and others 44 of the activity? 45

- Does the performer of the activity have a level of recognized authority, technical and 1 project management experience, and past record of success in the field of effort to 2 adequately accomplish the stated objectives? 3

VI.c. Number of Peer Reviewers 5 One or more qualified individuals with technical expertise in the appropriate discipline(s) should be 6 asked to provide peer review of informative scientific or scholarly information. Each review of 7 “Influential Scientific Information” and “Highly Influential Scientific Assessment” must include a 8 minimum of two independent peer reviewers. (See VI.k for procedures governing reviewer 9 selection.) 10

VI.d. Peer Review Structure 12 Peer reviews may be structured in various ways, ranging from one or more individual reviewers 13 (probably the most appropriate model for most NPS endeavors) to review panels or advisory 14 committees. The choice will depend on the novelty and complexity of the scientific information 15 and whether the desired result is individual reviews or a consensus among reviewers. In 16 determining the structure of a given review, the NPS must observe the requirements of the 17 Federal Advisory Committee Act (FACA), where applicable. If the review does not involve 18 convening a group of reviewers as a committee, the review is probably not subject to FACA. 19 Consensus panels may not be subject to FACA if established, controlled, and managed by an 20 outside organization, such as a contractor. 21

VI.e. Peer Review Manager 23 In general, reviews for information that will benefit from peer review but are not deemed 24 “influential” or “highly influential” will be managed by a park, region, or other office peer 25 review manager following the guidelines in this document. The peer review manager for 26 “influential” and “highly influential” NPS scientific or scholarly information activities will be 27 selected by the responsible Senior Executive Service (SES) manager. 28

VI.f. Required Disclaimer 30 The peer review manager must ensure that each product distributed for peer review under this 31 chapter is imprinted with the following disclaimer: 32

This information is distributed solely for the purpose of pre-dissemination peer 34 review under applicable information quality guidelines. It has not been formally 35 disseminated by the National Park Service. It does not represent and should not 36 be construed to represent any National Park Service determination or policy. 37

VI.g. Disposition of Peer Review Comments 39 The peer review manager must make all review comments available to the authors of the work 40 being reviewed, usually without identifying the reviewer’s identity. Authors may disagree with 41 peer reviewer comments on scientific or scholarly grounds. Where comments of a substantial 42 nature are not incorporated in a revision: 43

A. The author should document the decision not to incorporate the comments; and 44

B. The peer review manager should document appropriately the comments, how the author 1 responded to the comments the author used, the reasons for the author not incorporating 2 comments the author rejected, and the peer review manager's concurrence with the 3 decision. 4

VI.h. Peer Review Costs 6 Traditionally, peer review is carried out at no cost. The NPS recognizes that increasing demands 7 for reviewer time and associated travel may occasionally necessitate payment for reviewer time 8 and for associated travel, especially for review of “influential” and “highly influential” activities. 9 VI.i. Peer Review Confidentiality 11 Peer review must be conducted in a manner that respects confidential business information and 12 intellectual property. Reviewers must agree to not disclose or divulge any results or conclusions, 13 or make any public statements regarding the reviewed information before it is published and 14 released. 15 VI.j. Peer Review Documentation 17 Each peer review manager must maintain a written administrative record for all formal peer 18 reviews. 19

A. Documentation must be sufficient for an uninvolved person to understand the process 20 used and any changes made as a result of the review. 21

B. Peer review records will be appropriately maintained under direction of the responsible 23

SES manager, who should describe where and how long each administrative record 24 remains in the working files and identify the archives where the record will be stored. 25

C. At a minimum, the records must include: 27 The name and position of the responsible peer review manager; 28 The name, affiliation, and pertinent qualifications of each peer reviewer; 29 Applicable confidentiality and conflict of interest documentation for each peer 30 reviewer (see Appendix B); 31 The objectives and structure of the peer review; 32 A copy of all peer reviewer comments, and 33 Explanation of how peer review comments were addressed. 34

VI.k. Selection of Peer Reviewers 36

A. Peer reviewers must be true peers who are selected on the basis of their relevant scientific, 37 scholarly, and technical expertise and their objectivity. They must not be associated 38 directly with the work being reviewed. Reviewers should represent a range of viewpoints, 39 especially where legitimate scientific controversy exists. Reviewers should be able to 40 ensure that the information is effectively presented with the intended audience in mind 41 and be cognizant of controversial or high-visibility issues that may be relevant to public 42 policy; 43

B. Potential reviewers may be recommended by NPS staff members, scientific and 45 professional societies, members of the public, or authors of scientific and scholarly 46 products, but actual selection of reviewers rests solely with the peer review manager and 1 should be based on specific criteria for peer reviewers. Supervisors of potential NPS 2 reviewers are expected to encourage their employees to serve whenever feasible. 3

C. Peer review managers should use the widest range of sources of scientific, scholarly, and 5 technical expertise as is practical and appropriate for an information’s expected degree of 6 influence. Sources may include other federal and state agencies, Tribal governments, 7 professional societies, colleges and universities, private companies, non-profit 8 organizations, and the National Research Council, a part of the National Academies. 9

D. If standing panels are created to perform consistent peer review, membership should 11 rotate across the pool of qualified reviewers. 12 E. For all influential scientific information (including highly influential scientific 14 assessments), peer review managers must ensure that all reviewers comply with federal 15 ethics requirements and must adopt or adapt the National Academy of Sciences policy for 16 committee selection with respect to evaluating the potential for conflicts (e.g., those 17 arising from investments; employer and business affiliations; grants, contracts and 18 consulting income). The NAS policy is available at: 19 http://www.nationalacademies.org/coi/bi-coi_form-0.pdf. All peer reviewers for 20 influential NPS information products must agree to be bound by the strictest scientific 21 ethics and sign the NPS Conflict-of-Interests and Confidentiality statement (see 22 Appendix B). For scientific information relevant to specific regulations, the NPS shall 23 examine a reviewer’s financial ties to regulated entities (e.g. businesses), other 24 stakeholders, and the NPS. 25

F. For highly influential scientific assessments, the use of formal peer reviewers employed 27 by the National Park Service is prohibited unless the reviewer is employed only for the 28 purpose of conducting the peer review (i.e., special government employees). (This 29 restriction does not exclude informal, internal reviews in addition to the formal peer 30 review process.) 31

i. The only exception to this prohibition would be the rare case where the NPS 33 determines, using the criteria developed by the National Academy of Sciences for 34 evaluating use of “employees of sponsors,” that a premier government specialist is not 35 in a position of management or policy responsibility and possesses essential expertise 36 that cannot be obtained elsewhere. 37

ii. To be eligible for this exception, the specialist must be employed by a different park, 39 region, or office of the NPS than is disseminating the scientific information. The use of 40 such an exception must be documented in writing and approved, on a non-delegable 41 basis, by the Secretary or Deputy Secretary prior to the specialist’s appointment. In 42 addition, peer review managers shall avoid repeated use of the same reviewer on 43 multiple assessments unless his or her participation is essential and cannot be obtained 44 elsewhere. 45

VI.l. Peer Reviewer Anonymity 1 Peer review managers should clearly inform prospective peer reviewers whether – 2

A. Reviewer names will be disclosed and comments will be specifically attributed; 3 B. Reviewer names will be disclosed but comments will not be specifically attributed; 5 C. Reviewer names will not be disclosed but comments will be disclosed; or 7 D. Neither reviewer names nor comments will be disclosed. 9

To the extent information about a peer reviewer (name, credentials, affiliation) will be disclosed 11 along with his or her comments or analysis (see also Section VI.q), the peer review manager 12 must comply with the requirements of the Privacy Act, 5 U.S.C. § 522a as amended, and OMB 13 Circular A-130, Appendix I, 61 Fed. Reg. 6428 (February 20, 1996) to establish appropriate 14 routine uses in a published System of Records Notice. 15 VI.m. Standards Governing Peer Reviewers’ Conduct 17 The peer review manager must ensure that each potential peer reviewer has been asked to 18 disclose any personal information or situation that may create or appear to create a conflict of 19 interest. Prospective formal peer reviewers sign a statement disclosing any potential conflicts of 20 interest (see Appendix B). Each peer review manager must provide appropriate documentation in 21 the administrative record. Conflict of Interest statements are subject to the confidentiality policy 22 stated in Section VI.l. 23 VI.n. Public Participation 25 Whenever feasible and appropriate, the peer review manager must make a draft of each highly 26 influential scientific assessment available to the public for comment at the same time it is 27 submitted for peer review (or during the peer review process). 28

A. The office producing the assessment will add an explicit prefatory statement to the 29 disseminated material that states the material being disseminated to the public is 30 concurrently being submitted for peer review and, therefore, may be subject to change and 31 then reiterate this prefatory statement when sponsoring any public meeting where interested 32 members of the public can provide the peer reviewers with oral presentations on scientific 33 or scholarly issues of concern to the public. 34

B. When employing a public comment process as part of the peer review, the peer review 36 manager shall, whenever practical, provide peer reviewers with access to public comments 37 that address significant scientific, scholarly, or technical issues. To ensure that public 38 participation does not cause undue delay, the peer review manager shall specify time limits 39 for public participation throughout the peer review process. 40

VI.p. Peer Review Plans for Influential Scientific and Scholarly Information 42 Each SES manager will post a list of the manager’s Park, Region, or Office’s planned and 43 ongoing influential scientific and scholarly information along with associated peer review plans 44 on an Information Quality webpage accessible to the public. This webpage shall be updated at 45 least every six months. The webpage will provide contact addresses to allow the public to 1 comment on the adequacy of the peer review plans. Each peer review plan shall include: 2

A. The title, subject and purpose of the planned scientific information along with an 3 electronic link to the information (when available) and a contact person. 4

B. The timing of the review (including deferrals). 6

C. The structure of the review (e.g., individual letters, panel) including the number of 8 planned reviewers and the primary disciplines or expertise needed. 9 D. The method for reviewer selection and whether the public can nominate prospective 11 reviewers. 12

E. Whether there will be opportunities for the public to comment on the information to be 14 peer reviewed, and if so, how and when these opportunities will be provided. 15

F. Whether the agency will provide significant and relevant public comments to the peer 17 reviewers before they conduct their review. 18 VI.q. Peer Review Reports for Scientific and Scholarly Information 20 The peer review manager shall prepare a report that describes the nature of the review along with 21 findings and conclusions. The report shall either include a verbatim copy of each reviewer’s 22 comments (with or without specific attribution) or represent the views of the group as a whole, 23 including any dissenting views. The report shall contain the names of reviewers and their 24 organizational affiliations. Reviewers must be notified in advance regarding the extent of 25 disclosure and attribution. For influential scientific information (including highly influential 26 scientific assessment), NPS shall disseminate the peer review report (including the peer reviewer 27 information) on a webpage accessible to the public. In addition, where scientific or scholarly 28 information is specifically used to inform a rulemaking, the peer review report shall be discussed 29 in the preamble to that rulemaking and included in the administrative record for any related NPS 30 action. 31 VI.r. NPS Annual Peer Review Report 33 With respect to influential scientific information and highly influential scientific assessments, 34 each SES manager shall provide appropriate information for compilation into the Service’s 35 report to the Office of the Secretary by December 1 of each year. This information should 36 include the following: 37

A. The number of peer reviews conducted for influential scientific information and highly 38 influential scientific assessments; 39

B. The number of times alternative procedures (e.g., relying on refereed journal reviews) 41 were invoked; 42 C. The number of times waivers or deferrals were invoked (and in the case of deferrals, the 44 length of time elapsed between the deferral and the peer review); 45

D. Any decision to appoint a reviewer pursuant to any exception to the applicable 1 independence or conflict of interest standards, including determinations by the Secretary 2 under provisions of the OMB Peer Review Bulletin; 3

E. The number of peer review panels that were conducted in public and the number that 5 allowed public comment; 6 F. The number of public comments provided on the agency’s peer review plans; and 8 G. The number of peer reviewers that the agency used that were recommended by 10 professional societies. 11 The Associate Director, Natural Resource Stewardship and Science, will consolidate the SES 13 managers’ reports into the Service report and will submit the Service report to the Department. 14

VII. RESPONSIBILITIES 16

VII.a. The Director 17 The Director addresses appeals of responses to information quality complaints. The Director may 18 waive or defer some or all of the peer review requirements where warranted by a compelling 19 rationale. If the Director defers the peer review requirements before dissemination, peer review 20 must be conducted as soon as practicable. 21 VII.b. Associate and Regional Directors 23 The Associate and Regional Directors are responsible for ensuring that their staffs implement 24 these policies and procedures. 25 VII.c. Senior Executive Service Official Responsible for Conducting Scientific Reviews 27 The Senior Executive Service manager directly responsible for a park, region, or directorate will 28 designate an official (the SES manager) to have overall responsibility for implementation of 29 these guidelines. Where the responsibility for a specific action or actions may not be delegated 30 to a subordinate, this should be so stated. 31 VII.d. Project Manager 33 The project manager, as the overseer of an activity, applies these guidelines to determine whether 34 or not the activity requires peer review and, for an activity that does require peer review, is the 35 person who designates the peer review manager and who oversees implementation of these 36 guidelines with respect to the specific activity. 37 VII.c. Peer Review Manager 39 The peer review manager: 40

A. Determines what level of peer review is required; 41 B. Establishes the peer review process by setting objectives, structure, and timeframe for 43 completing the review, as appropriate, by: 44

i. Submitting the information to an appropriate refereed journal or other, credible, 45 professional peer review program. 46

ii. If the information is deemed “influential” or “highly influential,” conducting a 1 formal peer review using the process outlined in this guidance. 2

iii. If the information is not deemed “influential” or “highly influential,” 3 continuing a peer review process according to these guidelines. 4

C. Provides review findings to authors. 6 D. Ensures that the authors address comments adequately and fairly. 8 E. Ensures that proper records are kept. 10 F. Oversees independent entities or contractors commissioned to manage the peer review 12 process. 13 VII.d. Parks and Offices Responsible for Releasing Information to the Public 15 Parks and offices responsible for releasing information to the public must follow the 16 requirements of D.O. 11B by: 17

A. Ensuring that the information they release to the public, in any manner and at each 18 appropriate stage of information development, is developed from reliable sources and 19 meets Service standards for information quality. 20

B. Documenting the quality of all information that they release to the public, including 21 information on the internet. 22

C. Notifying formal complainants of the receipt of their complaint within 10 working days 23 of their receipt of the complaint from the Correspondence Control Unit. 24

D. Responding to formal complaints about information quality within 60 calendar days from 25 receipt of a complainant in CCU. 26

E. Responding accurately and appropriately to informal complaints about information 27 quality. 28

VII.e. The NPS Correspondence Control Unit (CCU) in the Washington Administrative 29 Program Center is responsible for: 30

A. Routing public complaints they receive about information quality to the information 31 disseminating office. 32

B. Reminding offices of their deadlines to respond to pending complaints forwarded by the 33

CCU. 34

C. Generating annual reports to the Department of the Interior of the number, nature, and 35 resolution of complaints received by the CCU. 36

VII.f. Chief, Office of Policy 37 The Chief, Office of Policy ensures that Servicewide information quality policies and procedures 38 are available on the web, including links to regional, network, office, and park information 39 quality web pages that are maintained by the respective officials responsible for conducting 1 scientific reviews. 2

VIII. DEFINITIONS 4

The following definitions apply to this document: 5 Conflict of Interest. Any financial or other interest which conflicts with the actions or judgments 7 of a reviewer because it (1) could significantly impair the individual’s objectivity or (2) could 8 create an unfair competitive advantage for any person or organization.