JOFOC_Attachement_1_-_Elsevier_Challenge_Reponse_Final_rcd.pdf

PURCHASE REQUST NUMBER: 89243118PSC000037 JOFOC Attachment 1

Comments Regarding Elsevier’s Capability Statement Submitted in Response to DOE’s Notice of Intent to Award a Sole Source Contract to Clarivate Analytics

OSTI subject matter experts (SMEs) have reviewed Elsevier’s responses to the requirements identified in OSTI’s Sole Source Award Notice. In many cases, the SMEs were unable to confirm or deny Elsevier’s claims in its response. However, there were instances where Elsevier appears to possess the ability to meet the basic requirement, but the method and manner by which Elsevier would implement those capabilities would likely negatively impact OSTI, which in the opinion of the SMEs is tantamount to not being able to be meet the requirement. Additionally, there were areas where Elsevier clearly did not present the requisite capabilities to satisfy OSTI’s requirements. Consequently, in preparing this summary of comments, OSTI has opted to only respond to requirements that it believes are clearly not met by Elsevier and cite “Other Points for Consideration” as areas that might be negatively impacted based on methods proposed by Elsevier.

OSTI Requirement: Must be able to provide author cited references for all full journal articles indexed from 1900‐present.

Elsevier Response: While article content goes back as far as 1788, Scopus currently provides cited references going back to 1970. Analysis of current article citations indicate that well over 99% of the cited references in currently‐published peer‐reviewed articles are to other works published in 1970 or thereafter. Elsevier believes it is better to make investments in continuous data quality improvement using current information technologies, including advanced semantic, machine learning, and NLP tools, than to spend resources on pre‐1970 citation indexing.

OSTI Response: Elsevier does not meet OSTI’s technical criteria of providing access to author cited references for all full journal articles indexed from 1900 to present. Elsevier only provides cited references going back to 1970. Two examples of how this would negatively impact OSTI’s program:

Item 10.1016/J.PHYSLETB.2012.08.020 on OSTI.GOV indicates that 12 of the 92 cited references are dated prior to 1970 (13%); item 10.1103/PHYSREVLETT.108.196802, indicates that 1 of 28 are prior to

1970 (4%). These percentages constitute a significant rate of missing information and would negatively impact OSTI’s mission‐critical goal of comprehensive electronic submission of scientific and technical information.

OSTI Requirement: Must provide APIs via the POST HTTP request method that support query searches of at least 21KB of text, with option to expand.

Elsevier Response: The Scopus APIs support POST requests with search strings that exceed 21kb. The

Elsevier team will work with OSTI to implement Scopus APIs to fit their needs.

OSTI Response: While the Elsevier response claims to support POST requests with search strings that exceed 21kb, their documentation at https://api.elsevier.com/documentation/ScopusSearchAPI.wadl only demonstrates support of GET method requests. If that is the case, Elsevier does not currently meet

OSTI’s technical criteria of providing APIs via the POST HTTP request method.

While OSTI appreciates Elsevier’s willingness to revamp their systems in areas where they do not currently meet OSTI’s complex requirements, the selected product must meet all technical requirements

PURCHASE REQUST NUMBER: 89243118PSC000037 JOFOC Attachment 1 upon award of the contract (April 1, 2018). Elsevier states they are willing to work with the OSTI team to implement Scopus Application programming interfaces (APIs) that fit DOE needs, and that they will work with us to ensure we can use POST for our specialized, large‐scale queries; however, capabilities that may be developed in the future do not meet technical requirements in the present.

Other Points for Consideration:

According to the Elsevier response to OSTI Requirements (see responses to OSTI requirements 3, 12, and

17), Elsevier is limiting indexing based on their determinations of relevancy. For example, per their responses, the Scopus Content Selection and Advisory Board members and machine learning determine what should be included in Elsevier result sets. These subjective relevancy determinations could reduce the traversal and citation display capabilities of OSTI.GOV and DOE PAGES if any items within OSTI’s system are deemed not relevant by Elsevier.

Also, the new Elsevier Funding Expansion may not match the depth of what OSTI currently has access to with the current vendor simply due to the age of the new Elsevier system (less than 3 months in operation). In OSTI’s own use cases, we find Digital Object Identifiers (DOIs) currently within our citation traversal system that do not pull citations from Scopus. This difference would negatively impact our ability to provide a comprehensive citation traversal system to DOE researchers. Elsevier is missing data that is currently available to OSTI, which impacts comprehensiveness and compliance reporting.

Consequently, considering the opinions of the SMEs, OSTI considers Elsevier as a source that is not capable of meeting OSTI’s requirements.