Ensuring quality as the basis of evidence synthesis: leveraging information specialists' knowledge, skills, and expertise
Year: 2018
Type: article
Abstract: Aeriform/Ikon Images/Science Photo Library The foundations of both Cochrane and the wider evidence synthesis community rest on the involvement of methodologists in the production of high‐quality systematic reviews. Epidemiologists, statisticians, and information specialists have had an enormous impact on the evolution of Cochrane and the advancement of systematic review methodology. Compared with other methodologists, information specialists tend to fly somewhat under the radar despite the fact that they act in a wide variety of roles. Information specialists organize, disseminate, and interpret information; they also preserve knowledge and ensure that it is discoverable in the future. The ‘science’ behind the profession – library and information science – is interdisciplinary and encompasses mathematics, logic, linguistics, psychology, computer science, operations research, communications, and management. Information specialists frequently work in libraries but are increasingly embedded in other organizations as information processes have become essential to most enterprises. In the emerging landscape of information overload and automation, information specialist expertise should be leveraged further to ensure quality and improve the efficiency of evidence synthesis. The basis for every sound systematic review is a carefully designed and well‐conducted search for evidence. Information specialists have traditionally safeguarded the quality of systematic review searches and ensured adherence to information retrieval methodology and reporting standards like MECIR and PRISMA.[1, 2, 3, 4] Information specialists have been central to the development of MECIR and PRISMA standards, but evidence shows that information specialists' skills remain under‐utilized in systematic review production, especially outside of Cochrane. An evaluation of 272 systematic reviews in 25 high‐impact journals showed that only 35% provided a partly reproducible search strategy.[5] Even for Cochrane Reviews evidence shows that search strategies are susceptible to error and benefit from peer review.[6] Another study of 630 general internal medicine systematic reviews showed that the level of information specialist involvement was significantly associated with search reproducibility. Information specialist involvement in systematic reviews improves reporting of search methods and also overall quality assessments.[2] Therefore, involving information specialists in traditional searching tasks will always be a core methodological recommendation for high‐quality systematic review production, but information specialists' expertise is increasingly being deployed in new ways. Nowadays information specialists are assuming editorial tasks, preparing and supporting the screening process and holistically managing the information underpinning a review.[7, 8] They share their knowledge of new tools for systematic review production, and they assist review authors in managing study selection and linking associated publications and records describing a study. Furthermore, The Lancet's REWARD campaign highlighted the pivotal role of information specialists in helping to reduce research waste by being embedded in biomedical research centres, journals' editorial teams, and funding bodies to scrutinize the rigour of search processes, assess the extent of uncertainty, and identify relevant ongoing research.[9] When information specialists elected to rename their job titles in Cochrane, from Trials Search Co‐ordinators, it was to recognize these evolving roles. The evidence synthesis community benefits from information specialists conducting methods research to underpin systematic reviews.[10, 11, 12] As investigators into systematic review methods, information specialists inform the conduct not only of reviews of effectiveness but also of prognostic and diagnostic accuracy reviews and reviews of adverse effects.[13, 14, 15] Information specialists expand the scope of included evidence in systematic reviews by searching trials registers, identifying non‐randomized studies, and using regulatory documents.[16, 17, 18] Identifying these types of documents poses new challenges, such as alternative approaches to search strategy design and study identification, as well as navigating and making sense of newly accessible clinical study reports and big data. As hard‐to‐search review topics grow in importance, information specialists contribute the expertise needed to help answer complex healthcare questions of importance to policy makers, practitioners, and patients. This necessitates adequate allocation of time and resources for information specialist research to ensure that systematic review methods can keep pace with the growing complexities of our stakeholders' information needs. As information managers, information specialists facilitate knowledge creation by supporting systematic review production, and they aid in the discovery of that knowledge by building metadata repositories. Information specialists can be found at the core of Cochrane's most innovative initiatives: Cochrane Crowd (community.cochrane.org/tools/data‐management‐tools/cochrane‐crowd); Evidence Pipeline (community.cochrane.org/tools/data‐management‐tools/evidence‐pipeline); and Linked Data (linkeddata.cochrane.org). Linked Data, a metadata project that aims to enrich Cochrane content, construct ontologies, and enhance interoperability with emerging information products, relies heavily on the expertise of Cochrane's information specialists. In addition, information specialists build, test, train, and implement machine learning for study identification, text mining for screening improvement, and objective approaches to improve consistent search strategy design.[19, 20, 21] These activities enhance the efficiency of systematic review production and are increasingly needed to realize new models of living evidence, such as living systematic reviews (LSRs). Information specialists' contributions to methods research on updating systematic reviews are critical to the development of LSRs.[22, 23, 24] LSRs are an emerging approach to continually updating systematic reviews at frequent intervals, incorporating new evidence as it becomes available.[25] Information specialists are uniquely placed to bring efficiency and rigour to the LSR process. They can determine not only when new evidence is available, but also when incomplete evidence, such as prospective trials register data or proceedings of ongoing research, should be replaced with completed study reports. They can also help to minimize the duplication of effort associated with having to screen search results more frequently. Given the longstanding responsibilities of information specialists to conduct surveillance activities and monitor the creation of new knowledge, living systematic reviews will be produced more efficiently if information specialists are involved to prevent the replication of flawed search methods, to reduce inefficiencies by inadvertent identification and screening of duplicate records, and to minimize the publication of ‘empty updates’ (i.e., updated versions of reviews that fail to identify new studies). We are witnessing a period of change in the sphere of knowledge production and management. Not only is evidence being produced at a faster rate than ever before, but predatory publishing activities are threatening academic scholarship standards.[26] Articles published in ‘predatory journals’ have usually not undergone a formal editorial process or peer review, which implies that systematic reviewers must review these data with even more scrutiny. As journal publications are predicted to be of diminishing importance as primary data sources for clinical trials, we must re‐evaluate methods of evidence retrieval and evaluation.[27] Future methods for generating any form of ‘living evidence’ will require the application of information specialists' skills to identify ongoing unpublished evidence; the proliferation of expedited publications of questionable quality will call on information specialists' knowledge of database indexing standards and journal bibliometrics to support the deselection of unreliable study evidence. To address changes in the information landscape, information specialists embrace automation, machine learning, and text mining. Information specialists also reduce methodological uncertainties by testing the validity of new tools and methods. As key contributors to evidence syntheses in evolving health systems, information specialists are key partners in the effort to enhance evidence use for a global community of patients, health professionals, and policy makers. As expert methodologists, information specialists drive successful innovations in information discovery, and stand ready to apply skills and expertise to improve the quality and efficiency of systematic review production.
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $5,100
OpenAlex ID: W2889972358