Defining Success in Open Science

Increased quality and efficiency of scientific outputs

Many participants believe that OS will curb the considerable amount of waste within biomedical research and development (R&D), lowering otherwise rising costs and providing a better return on investment than presently exists. In particular, funders and philanthropy expect that OS will lead to increased reuse of data and fewer ‘throwaway’ datasets that, once used by the data generator are afterwards virtually inaccessible. Biotech and Pharma cited the ‘reproducibility crisis’, noting an urgent need for more reliable academic outputs that can be used without lengthy in-house validation. They expect that OS will build open detailed knowledge of the basic biology and biochemistry of drug targets and pathways – outcomes that point to important downstream success factors, including the ability to rapidly select the most promising drug targets, to identify failure earlier in the innovation process and to reduce costly late stage failures – all contributing to lower attrition rates within the R&D pipeline. Additionally, Biotech expects that OS will lead innovation actors to concentrate their efforts where they can excel, reducing the redundancy of roles and activities: this should further contribute to the efficiency gains that many stakeholders anticipate from OS. Most participants across sectors expect OS to generate more, and more diverse, high quality datasets, together with the meta-data necessary to use them (including descriptions of methods, reagents, protocols and workflows, the instruments or platforms used in their generation, how and why data were collected).

Accelerated innovation and impact

Participants across sectors underlined innovation and public health-related factors as key outcomes of success. They expect OS to lead not only to faster innovation, but to deliver truly novel products and services that address unmet needs and bring measurable benefits to communities: “discovering tomorrow’s medicines, today”. Thus, government, philanthropy and national funding agencies expect improved health outcomes across their populations. Biotech and Pharma anticipate that OS will give rise to a greater diversity of research, penetrating research ‘white space’ and seeding novel research domains, including new interdisciplinary fields. Taken together, many participants expect OS to generate faster development of knowledge and its translation into products and services with marked social value.

Increased trust in and accountability of the research enterprise

Many participants agreed that increased trust is a key success outcome, and one that is more likely to be achieved through greater openness than by other means. Many expect that OS will augment transparency, and consequently instill greater accountability across the entire research process, including at the level of reporting on the use of public research funds and the resulting public benefits. Governments expect this information to foster public trust in the research enterprise, and greater appreciation, understanding and support of science. They anticipate that this will result in increased research participation, public funding of science and private donations. In parallel with heightened transparency, many participants said that a successful implementation of OS would require new and improved mechanisms to explain research to communities and transparent governance and technical mechanisms to ensure the security of sensitive or confidential personal data while facilitating legitimate and beneficial uses.

On their side, Biotech and Pharma expect OS to augment trust between innovation actors, leading to streamlined partnering and collaboration. Again, they cited the poor reproducibility of academic outputs and a tendency for universities to compete rather than cooperate, for example by over-valuing their IP or other research outputs, which can slow partnership or knowledge transfer negotiations and fuel industry skepticism. Here again, these players believed that OS could lead to a paradigm shift toward cross-sectoral complementarity and collaboration.

Increased equity in research

Participants highlighted increased equity as another key success factor that is most likely to be achievable by way of greater openness. They discussed equity and inclusiveness at the level of i) participation and individual agency in the research process; and ii) access to research outputs and benefits.

First, many expect that OS will foster democratization of the research enterprise, resulting in a greater diversity of people meaningfully involved and gleaning value from the process. Nevertheless, many delegates, including those from developing countries, noted that, to achieve these results, countries with the assistance of funders will first need to develop a sharing infrastructure that includes high-speed internet, as well as local research infrastructure and a critical mass of trained researchers. Bearing this in mind, many delegates believe that OS will lead to increased diversity of research leaders, collaborators and participants, including across communities of colour, gender, ethnicity and socio-economic group. Patient organizations expect OS to result in greater involvement of end-users and communities in the research process – for example, leading studies, framing research questions, making funding decisions and determining the outputs of value – and that there will be more funding available to ensure that these outputs are accessible to communities that participated in the studies. Many researchers expect OS partnerships to draw more clinical trials to OS research centres, augmenting local patient access to innovative therapeutics. Taken together, many delegates believe that OS will deliver more, and more immediate benefits, to communities from the research process.

Patient organizations also believe that greater openness will shift access and sharing decision-making to the individual donors, rather than researchers. Thus, OS will lead to new mechanisms to keep participants better informed about use of their materials and establish the individual as the ‘unit of openness’. Many participants expect OS will encourage other de-centralizations of power, including increasing collaborations in which developing country actors are equal partners or drivers of the research. In this regard, delegates underlined the need to avoid repeating historical power inequities whereby the benefits of some joint projects were coopted by the researchers from the more powerful or developed settings.

Second, by increasing access to knowledge, many participants expect OS to pave the way to increased scientific capacity in lower income, marginalized and developing communities. Again, they believe that OS will result in broader distribution of benefits, stimulating the development of research and sharing infrastructure, training, jobs and funding opportunities in lower income settings, and increasing retention of highly trained individuals in their local communities. Taken together, many participants agreed that heightened equity and inclusiveness through OS will bolster solidarity and justice, leading to greater empowerment of individuals and communities globally, and more opportunities to participate in the science innovation system, to create impact and to improve local health and well-being.

Better opportunities and recognition of early career researchers and youth

Many participants believe that greater openness will lead to development of new high-value jobs, and better and more diverse opportunities for students, post-docs and the next generation to launch their careers. For example, some of the new roles they foresee include novel positions and pathways in academia around data management, including data scientists, curators and stewards. They also expect OS to decrease barriers to students moving between academia and industry, by increasing collaboration and knowledge flow between the two settings. Many participants strongly underlined that at the very least, OS would not disadvantage early career researchers and youth who are considering entering the sciences. However, several noted that fears about the consequences for post-docs establishing their labs are a key reason that stakeholders may hesitate to embrace OS.

Positive economic impact

Many participants expect that OS will lead to equitable and positive economic impact. First, governments expect OS to prompt the private sector and venture capital (VC) to invest in research, where otherwise they would not. By augmenting OS public-private collaboration, OS is expected to increase the resources available to universities both through access to industry infrastructure and knowledge, and through additional funding. In particular, many participants believe that OS will lead to economic development in the communities housing OS research centres: in order for firms to take full advantage of the expertise, know-how and relationships embodied in local researchers and infrastructure, they will be obliged to set up in the environs, bringing jobs and investment. Thus, participants anticipate that OS will catalyze the development of vibrant local ecosystems, make launching start-up firms easier, and create more skilled jobs, and more jobs overall, at all levels. Industry and philanthropy further expect that OS will lead to the creation of new business models, including for VC and investment.

Implementation success

The Leadership Forum discussions often turned toward what will be needed for a successful implementation of OS. Many times, participants across the spectrum of sectors present stated that OS will require a paradigm shift in scientific research culture in order to realize its full potential. At the same time, they said that such a transformation would, to a large degree, show that execution of OS was well underway.

As noted in previous sections, Biotech, Pharma and governments said that successful OS implementation will result in better definition of the activities and roles of the various actors within the innovation system, including their specific responsibilities in the integration of OS.

Most researchers and industry believe that OS implementation will lead to an attitudinal shift amongst researchers in favour of sharing data and collaboration: individual researchers will come to view their outputs as part of a broader initiative to build a discovery platform for the benefit for all, rather than as belonging to them. Thus, successful implementation of OS will be characterized by researchers freely sharing data, publishing by default in open access journals and avoiding the use of restrictive IPRs: open practice will be fully integrated into every-day workflow by research institutions, governments and philanthropy. To aid in this, there will be many new resources available to practitioners, including training in how to conduct open practice and manage data, and tools, such as model workflows, sharing protocols and templates.

Governments and philanthropy noted that success will be reflected in the availability of long-term and sustainable funding to support OS infrastructure and more trusted open repositories for housing research outputs.

Many participants expect there to be an increasing number of data professionals, including scientists, curators and stewards, to ensure that data are managed and put to their best use. Likewise, many said that tracking of scientific outputs by DOI or other means will become standard. In parallel, participants agreed that a critical indicator of successful OS implementation will be the recognition of a broader range of outputs as publishable material by journals, funders and institutions – including reproducibility studies, datasets, policy publications, clinical guidelines, etc. – and the assigning of value to these in promotion, tenure and funding processes.