Visible to members of this folder

Promoting openness in professional advancement practices

Stacy Konkiel, Altmetric

Cheryl Ball, West Virginia University

Kim Barrett, University of California San Diego and the Journal of Physiology

Peter Berkery, Association of American University Presses

Jessica Clemons, University of Buffalo

Sheree Crosby, Cabell’s International

Holly J. Falk-Krzesinski, Elsevier and Northwestern University

In this report, we unpack how professional advancement practices—including and beyond promotion and tenure review standards—can be realigned to encourage researchers’ adoption of open access, open research, and open educational practices.

The remit of the Open Scholarship Initiative 2017 “Promotion & Tenure Reform” working group clearly connected researchers’ personal publishing choices to the oft-traditionalist system of promotion and tenure in the United States, wherein researchers feel compelled to publish in toll access journals or monographs if they wish to achieve tenure, win grants, receive awards, or otherwise advance professionally. Other professional advancement systems worldwide, such as university hiring and contract renewals and government and foundation grantmaking processes, similarly reinforce the primacy of toll access research formats. Hiring practices were of particular concern for our working group, given the increasing “adjunctification” and precarity of university posts in the United States. Due to these parallels, the Reform working group expanded our charge to consider hiring, grants, and other professional advancement scenarios common to researchers’ concerns worldwide.

Here, we set the scope of the current problem, discuss the reasons why professional advancement scenarios should be realigned to reward open research practices, identify challenges to reforming professional advancement scenarios wholesale and worldwide, recommend concrete actions for beginning the reformation process, and share resources related to professional advancement and open access.

Setting the scope of the problem

The workgroup initially grappled with the scope of the assigned problem. We were asked to develop a “widely-accepted and inclusive model....to help reduce the influence of journal publishing on promotion and tenure decisions and help make these decisions broader, more transparent, and less reliant on publishing and impact factor measures”. To a greater or lesser degree, not all workgroup members agreed with the underlying premise of this assignment as stated.

For example, in many disciplines, it is likely not desirable to reduce reliance upon publishing in journals per se, because peer-reviewed articles as a vehicle for reporting will almost certainly, and appropriately, remain the coin of the realm for academic advancement.

Instead, academia needs:

A closer reading of research by committees charged with evaluation, rather than relying on the surrogates of publication venue and impact factor;

A broader view of the types of scholarly outputs that committees should consider as evidence of productivity and impact;

An explicit acknowledgement of the benefits of publishing in open access venues; and

Incentives that encourage openness.

The final issue has relevance and benefits not only for the faculty member in question (greater visibility, increased opportunities for collaboration, and so on) but also for their host institution (particularly with respect to demonstrating the collective impact of its scholars, as well as fulfilling a commitment to making that work available to society at large -- this may be particularly important for public universities).

Early on in our discussion, our workgroup agreed that our work should simply be a starting point for exploration, as we were a small cross-section of stakeholders in academia. More stakeholders need to be at the table for developing full recommendations for practice. OSI is well-equipped and positioned to undertake exploratory research that can bring together a broad sample of researchers, funders, and chief academic officers to advance this agenda, as discussed in the Recommendations below.

We also wrestled with “mission creep”. Though many of the issues relating to openness are tangled up in other profound challenges in academia (e.g. What metrics do we use to evaluate research? How do overworked evaluation committees reward research quality over quantity? How does a researcher’s gender affect his or her ability to commit to collaboration, open research practices, and so on?), OSI must be careful not to get sidetracked in its mission to promote openness. On the other hand, we must be mindful that the issues inherent in infusing a culture of openness into academic advancement scenarios are likely highly dependent on discipline and culture, and some gains will be easier to achieve than others.

For example, this issue extends well beyond journal articles and/or STEM fields and the impact factors that almost exclusively apply in these settings, and yet it may be far more challenging to implement principles of openness in promotion and tenure practices for faculty in traditional “book disciplines” (i.e. the humanities and social sciences). The University of California’s open access mandate for work produced by their employees (including graduate students) is focused on “scholarly articles” and thus implicitly excludes books and monographs.

We should also consider the influence of changing norms and government mandates with respect to so-called “interim products of research”. For example, the NIH has recently revised its grant application guidelines to encourage reference to work that has not yet been published in a peer-reviewed venue, but that nevertheless can be made publicly available, such as pre-prints deposited to a public server. This change, which in part is intended to enhance rigor and reproducibility in research findings, could well impact the extent to which promotion and tenure, hiring, and other funding agency committees likewise consider pre-prints and other interim forms of scholarly communication in their deliberations.

Open practices, on all sides of the table

As our working group considered “openness” as a virtue in professional advancement scenarios, we quickly realized that open practices should be encouraged by all actors in the system.

Some examples of open practices that might be encouraged are:

Open researchers can publish their work open access, license their in-progress and completed work in ways that allows others to adapt and reuse it with attribution, “work out loud” to share their findings earlier on in the research process, share all of the outputs of their work (including research software, data, notes, presentations, and other “non-traditional” formats), and commit to public engagement, to connect other researchers and the public with their work. Due to their relative career security, senior researchers are especially well-positioned to be advocates for open research, as well.

Open educators can share their educational resources openly, use others’ open educational resources in their curriculums, collaborate in the open to develop teaching materials, and encourage their students to develop in their own right as open researchers.

Open employers can offer transparent hiring and retention guidelines for faculty, share the benchmarks and metrics by which faculty and departments are evaluated, be explicit about professional advancement expectations, and make their promotion and tenure evaluation guidelines freely available in open formats, so that other institutions might learn from their examples.

Open funders can similarly create transparency in the evaluation process, freely sharing the guidelines used to evaluate funding proposals, encouraging researchers to share their grant proposals (both accepted and rejected), and (where appropriate) releasing more information on the review process used by their committees.

These various examples boil down into two main facets of openness: openness in expectations and evaluation practices, and openness in the production of research.

Challenges to large-scale change

Our workgroup also cataloged barriers to increasing openness in P&T review processes, both with respect to the openness of the process itself, and with respect to encouraging and rewarding faculty for following publishing practices that increase the accessibility of their work. Many of these challenges also apply to making changes to the way annual review and reappointment processes are managed, hiring is done, and grants are awarded. The challenges discussed could be categorized as both structural and cultural. These identified challenges inform our Recommendations below.

Academic freedom is by far the largest issue to consider in promoting change, especially in the United States. Changes to P&T criteria that are intended to reward openness should not infringe on the rights of an individual faculty member to decide where his/her work should be most effectively published. At the same time, academic freedom can protect researchers’ rights to make their work open access, even where review committees and other researchers disagree.

There is also an acute need to understand the level at which the P&T process is controlled at various institutions. For example, does a department or other academic unit regulate who can go up for tenure and what the tenure requirements look like, or is this centrally defined by the Chief Academic Officer, faculty senate body, or Board? Do faculty have a right to be reviewed at given checkpoints in their career, or little or no independent agency with respect to presenting their credentials? What are the various institutional levels of evaluation (departments, schools/colleges, provost, etc) and are they simply advisory or determinative in the process? Answering these questions across a range of institutions will define the locus at which efforts to increase openness must be directed.

There is also a need to define the landscape for openness in P&T and other academic advancement processes. The extent to which openness is already supported beyond institutional OA mandates is unknown. P&T documents that were reviewed were found to be largely opaque with respect to the extent to which openness and accessibility are valued for professional advancement (with a few notable exceptions as referenced below).

There is also a hurdle to be overcome in defining how best to reach faculty to help them describe their impact through openness. Disciplinary bodies or societies could play an important role in codifying researcher norms and expectations, which in turn could guide the policies of academic institutions with regard to rewarding openness.

These researcher education efforts would also inform those who are asked to supply outside letters in support of professional advancement. Because such letters carry disproportionate weight in most P&T processes, it is important that no researcher should ever again have their professional advancement endangered by a letter from an individual who focuses unduly on the “quality” embodied by a given publishing model or venue (especially those who incorrectly conflate open access journals with low quality).

There are also cultural and resource issues that may limit faculty enthusiasm and even practical ability for making their work open, such as gender, the digital divide, and variable access to both financial resources and the technology required to fully open one’s scholarship.

Faculty working in disciplines that offer ample support (e.g. staffing, time allowances, financial resources) and/or which explicitly value or even require openness are likely to progress more rapidly towards developing professional advancement scenarios that reward openness. But many others work in areas or institutions with neither the funds nor the technological infrastructure to support such openness. Realigning institutional support with openness goals will be crucial to allowing researchers the ability to make one’s work open.

For example, in most institutions with which workgroup members were familiar, institutional funds set aside to assist faculty in paying APC’s have been woefully disproportionate to the size of the research enterprise. Likewise, even within disciplines that are well-funded overall, these funds are by no means evenly distributed. Moreover, some types of research are inherently more costly, leaving faculty with little in the way of discretionary resources to underwrite OA publication or cover deposit fees for data archives, or even for institutions themselves to support OA, “free to researchers” initiatives like ArXiv or the Open Library of the Humanities.

Recommended work moving forward

Over the coming months and years, we recommend that OSI take forward the following projects to better meet community needs and increase stakeholder buy-in.

Research the existing landscape to better understand open research recommendations and requirements in professional advancement materials (P&T guidelines, job advertisements, university contracts, annual appraisal guidelines, etc) at leading universities worldwide. Estimated 2-3 months completion time required. This might include research into:

The extent to which open access publishing and other open research practices (data sharing, public scholarship, etc) are encouraged and encoded in existing promotion and tenure guidelines and job advertisements.

The University of Illinois at Urbana-Champaign convened a workshop on the University of the Future with a focus on open scholarship in March 2017. Group 1 focused on issues of credit & attribution. The group's preliminary report states, “Much of the discussion focused on the need for the reform of assessment systems and respect for a diverse range of outputs products and activities. It also became apparent that there are different types of credit; we need to understand how credit and attribution is different for distinct open scholarship communities.” Partnering with this and similar groups to explore attribution as applied to the research evaluation process will be important to the development of a framework for promoting openness across all academic disciplines and sectors.

It will also be important to investigate how current evaluation practices stack up against our vision for a more open Academy. Juan Pablo Alperin (Simon Fraser University, Canada) and Erin McKiernan (National Autonomous University of Mexico) is currently investigating the former topic and has indicated that he would be happy to advise OSI on his findings, once complete. OSI 2017 keynote speaker Keith Yamamoto (University of California San Francisco, USA) indicated that his university was in the process of radically revising their P&T guidelines to incentivize better research practices; OSI might use Yamamoto’s efforts as a case study, recommending certain practices to similar universities, or feature other universities like the University of British Columbia or IUPUI, who have already enacted more “open” P&T guidelines (see the Resources section for more information). OSI could also research—or fund independent researchers to investigate—the extent to which openness is addressed in academic job listings.

How universities might increase transparency in promotion and tenure process. Our group agreed that openness works on both sides of the hiring and review table, and that many institutions and departments suffer from a lack of clarity and transparency for what is expected of promotion and tenure candidates. By making the process more transparent—and by explicitly including encouragement of open research practices in promotion and tenure preparation and evaluation guidelines—we can make it easier and more appealing for researchers to practice open research.

How bibliometrics and other metrics used in research evaluation can encourage (and discourage) open research practices. For example, rather than relying on rather than relying on journal-level metrics as proxy for understanding research quality, one might rely upon article-level metrics, including citations and altmetrics. Similarly, counting data and software citations towards promotion and tenure might encourage more researchers to share their data and research software, especially in ways that encourage reuse.

Engage scholarly societies and high-level university research administrators and provosts to learn more about the challenges of promoting openness in promotion and tenure from their perspective. Estimated 18 months or more required. Areas for investigation include:

Which scholarly societies already promote openness in their best practices for promotion and tenure (similar to the Modern Language Association’s recommendations for evaluating digital scholarship, see below);

Pressures that drive chief academic officers/provosts at leading universities worldwide. In turn, we expect that this information will help us position any future OSI programs that encourage changes in university-wide promotion and tenure practices that are decided by senior academic administrators;

Articulating the benefits of Open Access to decision makers who would be in a position to adopt Openness in promotion and tenure principles (department heads, provosts, etc);

The feasibility of incorporating policies that encourage open research practices into university and department accreditation processes;

Development of model policies and guidelines, which societies, universities and departments can easily adopt (akin to the use of model legislation in the United States, where groups promote a law or policy that is vetted by experts and adopted by state legislatures (e.g. Uniform Electronic Transactions Act, which makes voting via email legal for non-profit boards).)

By actively engaging with powerful stakeholders at universities and scholarly societies worldwide, we can better address concerns over the costs and benefits of adopting policies that encourage openness. Through the development of model policies and guidelines that encourage open research practices, we can also reduce the friction of developing and passing local policies and guidelines.

Most debate around open research practices and professional advancement only address STEM use cases. OSI delegates should conduct a thorough literature review and interview and survey faculty from across all disciplines, career levels, and institution types to understand:

Where are the pain points for researchers with respect to Open Access and open research practices?

How many researchers worldwide are beholden to OA and open research mandates? What are the pain points for those researchers?

How do institutional OA policies impact tenure-track faculty that are also required to follow promotion and tenure requirements that disincentivize open research practices?

Do funder requirements for Open Access positively affect open research practices in the tenure and promotion process, where such P&T requirements weigh research funding into P&T cases?

What can we learn about researcher evaluation from research institutes or academic libraries that don’t have tenure (e.g. Scripps or HHMI)? What are the best parts of research evaluation practices worldwide, which we can borrow from to promote openness? What are the worst evaluation practices that should be avoided?

Estimated 6 months completion time required.

When enough intelligence is gathered from all stakeholders to make concrete recommendations, we suggest that OSI develop a plan for the following:

Presenting recommendations and model policies and guidelines to senior academic administrators and department chairs of all disciplines, from a cross-section of universities worldwide. OSI should clearly articulate both the potential benefits and challenges of introducing such recommendations.

Gathering and incorporating initial stakeholder feedback into recommendations.

Assembling a pilot program for enacting revised recommendations, in partnership with scholarly societies, senior academic administrators, and department heads worldwide;

Developing a final set of recommendations that offers concrete plans for encouraging adoption among various disciplines, scholarly societies, and universities worldwide. OSI may wish to engage the change-makers (e.g. department heads and Chief Academic Officers) and precedent-setters at organizations like Association of American Universities and Association of American University Presses.

Resources and Guidelines

Promotion & Tenure

Here are known promotion and tenure guidelines that address open research practices:

Indiana University Purdue University Indianapolis

University of British Columbia

Other universities and institutes that have endorsed open research practices in other ways include the Montreal Neurological Institute and Hospital at McGill University and Université de Liège (which requires the deposit of research into an open access repository as a precondition for evaluation and has elsewhere publicly endorsed Open Access). The Open Access Tracking Project collects resources related to openness in promotion and tenure.

Job Postings

A number of job postings in the sciences that are explicit in their desire for open researchers can be found on the Open Science Q&A website. The Open Access Tracking Project also collects job postings that are related to open access or that consider open research practices. A group of researchers is also developing an Open Hiring Policy rubric, which institutions can use in their own hiring practices.

Funding Agency Policies

In many disciplines, the ability to win research funding is linked to one’s promotion and tenure evaluation. Here are some funding agencies that explicitly call for open access or other open research practices in their granting guidelines:

Wellcome Trust

Research Councils UK

National Institutes of Health (US) data sharing policy

MIT also maintains a list of US federal funding sources that have open access or open data policies.

Precedents from the realm of “digital scholarship”

The committee recommends that any guidelines on openness look to precedent guidelines on the recognition of digital scholarship and “non-traditional” research formats:

Modern Language Association guidelines for evaluating digital scholarship

Conference on College Composition & Communication Work with Digital Technology

American Historical Association guidelines for evaluating digital scholarship

PraxisWiki’s Resources for Evaluating Digital Scholarship includes many more links to institutional and scholarly society guidelines on evaluating digital scholarship

Acknowledgements

The authors wish to thank Daniel Mietchen, Lisa Hinchcliffe, and Dorothea Salo for their substantive and valuable feedback on this report.

Contributions

Writing – Original Draft Preparation: SK and KB

Writing – Review & Editing: HFK and JC

Conceptualization: CB, KB, PB, JC, SC, HFK, SK

Promoting openness in professional advancement practices
Setting the scope of the problem
Open practices, on all sides of the table
Challenges to large-scale change
Recommended work moving forward
Estimated 6 months completion time required.
When enough intelligence is gathered from all stakeholders to make concrete recommendations, we suggest that OSI develop a plan for the following:
Resources and Guidelines
Promotion & Tenure
Job Postings
Funding Agency Policies
Precedents from the realm of “digital scholarship”
Acknowledgements
Contributions