Technology on Crossref

Investing $4.9 million of our surplus in rebuilding the Crossref system

Helena Cousijn — Wed, 12 Aug 2026 00:00:00 +0000

The infrastructure that registers and distributes Crossref metadata has been running and growing for more than two decades. It now does considerably more than it was built to do, and it shows: two large, tightly coupled monoliths (Content System and REST API), where a change in one place risks breaking something in another.

In its July meeting, the Crossref board approved the use of Crossref surplus funds for a three-year project to accelerate the redesign of the Crossref system. Our plan is to use $1.6 million a year for three years (~USD $4.9M in total) from our surplus funds, which will help us set up a modern infrastructure consisting of separate services for the different functionalities that the Crossref system offers.

This infusion of time-limited resources will augment our existing team. We will continue to maintain the current system, add small improvements, fix bugs, and make schema changes. We’ve mapped out our time and capacity to ensure that we can keep the current system active and robust while building the new system at the same time. In the past, when we’ve attempted large redesign initiatives, we’ve done a code freeze on the existing system—we won’t be doing that this time. 25,000+ members and thousands of downstream services rely on the system daily, so we will continue to provide time and energy to support that while building for the future. As always, you can view our public roadmap to see progress with what we’re working on.

Background

Over the years, the board, leadership team, and staff have discussed the need to address long-term technical debt in the Crossref system and the need to move from a monolithic codebase to a more modern, manageable, resilient technology that recreates the Crossref system. We are aiming to do this through a set of connected services and functions that better meet our mission and enable the Research Nexus, our goal of a rich and reusable open network of relationships connecting research organisations, people, things, and actions.

Our colleagues Paul Davis and Sara Bowman described some of the current challenges and plans earlier this year in Hit refresh: redesigning our technical infrastructure. We migrated from a closed-source database to PostgreSQL in 2024, moved out of a physical data centre into the cloud in August 2025, and have steadily been deprecating older tools not fit for purpose, such as Event Data, Co-access and ‘old’ Metadata Manager while rebuilding replacements. That was some of the groundwork; now we’re rebuilding the centre.

The two largest components of the current architecture are the Content System (CS) and the REST API. Both function as monoliths. They use mono-repositories, and their internal components are tightly coupled. Over time, they have accumulated complexity, substantially expanding beyond their original purpose. Each of them is now responsible for many different functions, with different needs and characteristics. This monolithic and tightly coupled architecture makes it difficult to maintain, scale, or expand our system. Adding new functionality often requires workarounds, which further increases complexity and adds technical debt.

As a not-for-profit, any surplus we generate has three possible destinations: reserves, paying down debt, or reinvestment in the mission. We met our goal of holding twelve months of operating costs in reserve in 2023, and we carry no debt, so reinvestment is where the rest belongs. We aim to generate surpluses to ensure that we are operating within our means, but we don’t need to accumulate them.

Some of that reinvestment goes outside Crossref; this year we’ve supported other community initiatives and infrastructure that people rely on, totalling around $500,000. A large proportion of the rest should go where members will feel it.

Reinvestment also has to be the right shape. Under the Principles of Open Scholarly Infrastructure, time-limited funds are used only for time-limited activities: day-to-day operations are covered by day-to-day revenue, and one-off funds go to one-off projects. A three-year rebuild, staffed by fixed-term roles embedded in existing teams, meets that test.

What will we rebuild?

At the end of 2024, we introduced a new structure with Crossref work happening within three cross-functional programs: Co-creation and Community Trends (CCT), Contributing to the Research Nexus (CRN), and Open and Sustainable Operations (OSO). The redesign of the Crossref system will also happen within these three programs. The approach we’ve taken is that we’ve identified the different functionalities of the current system, and divided these between the three programs. In the tables below, you can see which services will be rebuilt within each program.

Co-creation and Community Trends (CCT)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Internal APIs for UIs	Purpose-built backends to serve various user interfaces. This is an architectural approach that we have recently begun using to build the new Metadata Manager record registration tool.	Take data from sources such as the Postgres database or the data lake and provide it in a form that user interfaces can use.	This approach will allow us to decouple existing interfaces from the legacy system codebase and build new ones as part of the other projects listed below.
Member self-service	A new interface for members to perform routine tasks related to their member account and user management.	Currently, these tasks are performed in highly manual processes by the Membership team. In some cases, there are semi-manual processes in place, such as with title transfers, where members can fill out a request form on our website.	A self-service portal powered by the new authentication and authorisation service (mentioned below) will lighten the workload of the Membership team. For example, we estimate that about 15% of member support tickets in 2025 represented contact updates which had to be handled manually.
New Admin tool	A user interface for members to perform routine tasks related to their DOI records and metadata submissions.	Members can sign in and upload metadata submissions or queries in XML format, as well as view details about their past submissions. The current admin tool lives at doi.crossref.org.	We will replace the current Admin tool with an interface that integrates with the new authentication and authorisation service and uses interstitial APIs for its functionality. The new interface will also be more accessible and usable.
ORCID Auto-update	A workflow without a user interface or authentication with the Crossref system.	Auto-update pushes metadata about works in the Crossref database to the ORCID API and asks researchers’ permission for Crossref to push further updates and additions.	The main goal of rebuilding our ORCID Auto-update is to make it a standalone service, less entwined with the content system. We will also tweak functionality that we have been unable to make in recent years.
Reports	A series of tables and dashboards presenting metadata from the Crossref database, including DOI resolutions, conflict reports, error reports, and participation reports.	Most report interfaces are openly available via the Crossref website. They are used mostly by members to self-serve information about their metadata on the member, prefix, or title level.	We will change the data source of these reports to the data lake and unify their overall architecture. On the frontend side, there is much scope for improving the usability and accessibility of these interfaces for greater analysis by members and the community.

Contributing to the Research Nexus (CRN)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Funder matching	Matching member-submitted funder metadata to a funder identifier.	For works with funders and no funder identifier, match the funder name to a funder registry entry.	Build a new matching architecture and match funder names to ROR identifiers. Matches are added directly to the REST API as asserted by Crossref.
Preprint matching	For registered journal articles, check if there is a corresponding preprint it can be linked to.	Looks for an exact match between the title and authors. Currently notifies the preprint member via email who is obliged to add the relationship to their metadata (which Crossref reciprocates from journal article records).	Research and test a new method for matching. Add matches directly to the REST API, marked as asserted by Crossref.
Reference matching	For references deposited without a DOI, look for a corresponding Crossref DOI and add those to the metadata (see our recent update on the scale of this service)	Several different processes are used, depending on the completeness of the deposited metadata. Matching is carried out during the deposit process and added to the XML record.	Research and test a new and further improved method for matching. Add matches to the REST API, marked as asserted by Crossref.
XML API	A public API to deliver metadata in XML format.	Use a DOI to look up metadata or construct an XML query.	Reimplement separately from the deposit system. Re-evaluate the features.
Metadata enrichment	Supplement member-submitted metadata with other information held by Crossref.	Crossref metadata records have `information` added during deposit, including member information, citation counts, and relationships.	Review current processes and identify those that can be separated from metadata submission. Where possible, build standalone services that create enrichments, preparing for additional third-party data sources.

Open and Sustainable Operations (OSO)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Authentication & Authorisation	An identity management system	Authenticate and authorise users to access the different Crossref tools and services.	Improve security, allow for member and sponsor self-service to manage users and permissions across their teams and vendors.
Billing	A service to generate invoices and reports based on new record registrations per quarter.	Generate a .csv of billable submissions and upload to our accounting system to generate invoices.	Reduce technical debt and remove code from the legacy system; allow for point-in-time billing.
Notifications	A service to send emails or callbacks to members as new DOI records are registered or existing ones updated.	Transactional notifications on submission status. Additional notifications when works are cited by other works.	Reduce/eliminate the number of emails sent, reduce technical debt, remove code from the legacy system.
Handle system	Update and look up DOI in the Global Handle Registry.	Registers the DOI string and associated landing page URL for redirection (DOI resolution).	Implement better fallback and error handling to make connections with the Global Handle Registry more resilient.
Deposit system	System that receives XML deposits from members.	Processes deposits and makes them available to other systems, like the REST API.	Separate out the different components of the deposit system.

The three-year timeline refers to the duration of the additional investment that is now being made. We don’t expect all of the rebuild work to be complete within three years. And system maintenance and improvement is never done. The additional resources will accelerate our ability to address these projects and get them substantially complete or meaningfully underway within three years. We will start with the things that are easiest to separate out of the current system, and work our way through it. The deposit system will be the last part we’ll tackle— after the initial 3-year period. Exactly how we’ll break out or couple together its components will depend somewhat on how the rest of the work in all three programs proceeds.

Additionally, there will be work in the OSO program to pull out deprecated/unused code from the content system as other programs build new free-standing services.

Of course, an end goal is to rebuild all of this in open code repositories, and eventually turn our amber-coloured ‘open source’ to green in our biannual POSI audits.

What does this new investment and approach mean for our community?

Having seen the long list of things we’re planning to do, you may be wondering how this will impact you. Here are some things you should know:

You’ll be able to use our services throughout this project. Part of the reason why we are rebuilding different functions one by one is that we don’t want to interrupt the services we offer to our growing membership and users. The current services will continue to run until the new ones are in place, and we’ll have clear transition plans and documentation in cases where you need to update your workflows.

We won’t just rebuild like-for-like, but we’ll make improvements when we rebuild services. We want to continue to provide the functionality we currently offer to our members, but we also realise there are things that could be better. The clearest example of this is member self-service, which will be one of the main priorities for the CCT program next year. For example, as mentioned above, in 2025, about 15% of member support tickets were contact updates: changes members should be able to make themselves in seconds, but which our membership team handles by hand; that is the kind of thing that will improve life for both members and staff.

We are planning to deprecate some older/less-used services along the way, but we’ll give you plenty of notice. Crossref is probably running more services than you realise! We’re therefore evaluating all our services as part of the redesign, and keeping only the ones that are actively used and really add value for the community. If we decide to retire a service, we will contact all users so they have enough time to transition to an alternative service. Everything we’ve deprecated so far is also itemised on our website.

Community input will be a big part of the process. To ensure the services we offer meet your needs, we will need to keep talking with you. Our community managers and UX researchers will be reaching out to users of different services to understand use cases and ensure these continue to be met within the new system.

This blog is the first tagged ‘system-redesign’ so follow this category for ongoing updates. If you have feedback or questions, please head to the dedicated community forum space for system redesign and ask us in the open.

Two halves of the same decision

Later this week we’ll also announce the latest round of fee changes resulting from our Resourcing Crossref for Future Sustainability (RCFS) work. What we charge, and what we do with what we collect, are two halves of the same decision; fees are to come down at the same time as we commit the largest single investment in our infrastructure we have ever made. Both follow from the same principle: we set fees to cover the cost of running the infrastructure, and when we hold more than we need, it goes back to the community and into the longevity of the scholarly record.

Our recent position paper on PIDs in research infrastructure policy argues that the value of an identifier lies in the metadata and services around it, and in whether those can be maintained over the long term. This approach to futureproofing open scholarly infrastructure is an example of that maintenance in action.

Acknowledgements

We would like to thank Sara Bowman, Carlos del Ojo Elias, Bharath Govindarajan, Stewart Houten, Martyn Rittman, Lena Stoll and Patrick Vale for their work on the system investment proposal and contributions to this blog post.

Mission Accessible: building better user interfaces for everyone

Lena Stoll — Thu, 21 May 2026 00:00:00 +0000

Today is Global Accessibility Awareness Day, and accessibility has been on our minds lately. We’ve recently completed an internal audit of all our user interfaces, and have added a new accessibility page to our website, where you can find the accessibility documentation that we put together as part of the audit.

Why accessibility matters

Of course we want to keep the barriers to participation in Crossref as low as possible for users with various disabilities. But also, more accessible tools work better for everyone. A person’s access needs can change really quickly: even if you consider yourself to be relatively able-bodied, you are only one minor inconvenience away from at least a temporary disability. All it takes is some dazzling sunlight hitting your eye or your phone screen, or perhaps your dog going after a rabbit in an awkward direction while you are holding the lead (ask me how I know!) - and before you know it, you will be relying on accessibility features to navigate the digital and/or physical world for a while.

An accessible user interface is one that you can navigate and interact with by various methods, including a mouse or touchpad, keyboard, screen reader, voice control, and other assistive technologies. It can be used on various screen sizes and supports zooming in or out without losing any content or functionality. It has sufficient colour contrast, doesn’t flash fast-moving images at you, and has a clear structure that can be understood by both humans and machines.

Where we are today

It is worth mentioning that we didn’t only start thinking about accessibility when we started tackling the full audit of our user interfaces in March 2026. For example, Patrick Vale has previously written in this blog about a browser extension he has created to improve the accessibility of DOI links anywhere on the Internet. And we have known for a long time that there were accessibility gaps in many of our tools, but we didn’t have this centrally documented anywhere.

When we did begin testing all our interfaces for compliance with level AA of the Web Content Accessibility Guidelines (WCAG 2.2) as part of the audit, we knew that some of what we would find was not going to be pretty. In the 26+ years of working with and for the scholarly community, Crossref has built countless tools and reports to offer to members and users, many of which we still maintain today. These are often decades old and have been built in a way that makes it virtually impossible to make them more accessible without rebuilding them entirely. So we know that we will continue to have accessibility gaps for the foreseeable future, but at least now we have a better idea of the scale of the challenge.

It’s also not all doom and gloom: more recently created user interfaces, such our new Metadata Manager, performed much better in the audit than legacy alternatives such as the web deposit form. We found a similar trend when looking at our report interfaces. To illustrate this, compare what happens when running the axe DevTools extension for Google Chrome on a member’s participation report - this is a user interface that was completely re-implemented in 2025. Doing this brings up 26 issues:

Meanwhile, the browsable title list, which has completed a few more trips around the sun, has 254 issues listed:

Beyond WCAG

If you’ve read this far, I hope you are convinced that accessibility is more than just ticking boxes on a conformance report. But especially for a global community like ours, there are other, less technical barriers to participation that we have to consider. For example, language is a major accessibility factor: much of what we as Crossref staff write and say is in English. When we host community events, we enable captions, and we try to leave space for these captions at the bottom of our slides.

We have also started experimenting with simultaneous interpretation during our online events, such as our recent project showcase event for the 2026 metadata sprint in São Paulo. You can find recordings of this event in Spanish, Portuguese, and English on our YouTube channel to see the promising results of these efforts.

What we are working on next

We are currently addressing the accessibility issues identified in our audit of the Crossmark service. Many Crossref members have implemented the Crossmark button and pop-up on their own platforms and websites, so we thought this was a great place to start the remediation efforts following our audit.

We are also in the process of redesigning our main website, www.crossref.org, following an information architecture review completed in 2025. Making changes to the design and navigation of our website will be the perfect opportunity to make our content not just more discoverable and more understandable, but also more accessible.

Clearly there is even more to be done, so watch this space for more updates on our accessibility roadmap and improvements. And if you have first-hand experience of using Crossref services and interfaces with assistive technologies, or you have other input or feedback you’d like to share, leave a comment below or start a discussion in our community forum.

References

Vale, P. (2025). Enhancing DOI Accessibility for All Users. Crossref. https://doi.org/10.64000/pp4rw-mtv44
World Wide Web Consortium (2024). Web Content Accessibility Guidelines (WCAG) 2.2. Retrieved May 8, 2026, from https://www.w3.org/TR/WCAG22/
Montilla, L. & Mahomed, R. (2026). Voices from Crossref Metadata Sprint in São Paulo. Crossref. https://doi.org/10.64000/a5qzf-k1738
Stoll, L. & Korzec, K. (2025). Request for proposals: Crossref website information architecture review. Crossref. https://doi.org/10.64000/058mr-k3s56

Enhancing DOI Accessibility for All Users

Patrick Vale — Mon, 28 Apr 2025 00:00:00 +0000

2025 Update

In 2022, we set out to update our DOI display guidelines with the intention to adopt the proposals in 2025. It’s important to note from the outset that we are not mandating any immediate changes to the DOI display guidelines. Instead, we are working with our community to co-create a solution that addresses the diverse needs of all users, rather than imposing technical changes that may not suit everyone.

Background

DOI links are the lifeblood of scholarly communication. They’re the canonical identifiers that enable researchers to find, cite, and assess academic work. In essence, they’re stable, reliable, and easy to use—provided you can see them. But what happens when a user can’t rely on visual cues?

The Accessibility Challenge

For users of screen readers and other assistive technologies, the full value of a DOI link can be lost. While sighted users benefit from the context surrounding a DOI link—such as the title, abstract, and other metadata—screen reader users often hear just the bare URL. This means they might not know what content the DOI link represents, leading to confusion and a diminished browsing experience.

The problem is compounded by the technical nature of DOI links. Being URIs (Uniform Resource Identifiers), they don’t naturally lend themselves to the same accessibility techniques as standard URLs. When we attempted to tweak DOI links directly, every change that improved accessibility for one group inadvertently hindered another. Whether it was a WCAG (Web Content Accessibility Guidelines) rule or an ARIA (Accessible Rich Internet Applications) attribute, a solution that worked in one area would break in another.

A Community-Driven Approach

Realizing that a one-size-fits-all fix wouldn’t work, we took a different approach - one that involved the community from the outset. After consulting with early adopters and attending an insightful session with the JATS4R accessibility group, it became clear that the answer lay in experimentation and iteration. Rather than modifying the DOI display guidelines immediately, we are developing a tool that enhances the user experience without disrupting the current standards.

It’s worth noting that this solution places the responsibility on the end user rather than on publishers and platform providers. However, by doing so, users can have a consistent browsing experience regardless of the platform they use to access scholarly content. This approach also serves as an important stepping stone toward a future publisher-provided solution—be it via accessibility-focused JavaScript or a mandated dual-link implementation—and any efforts to recommend or mandate such changes will benefit greatly from concrete evidence of the effectiveness and scalability of this approach.

Introducing the DOI Accessibility Enhancer

First demonstrated at the recent Crossref Annual Meeting, here we share our DOI Accessibility Enhancer browser extension. Available now on the Firefox Add-on Store and the Chrome Web Store, this extension is designed to improve the experience of DOI links for screen reader users without altering the default behavior for sighted users.

How It Works

Scanning for DOI Links: The extension scans any webpage for DOI links.
Querying Metadata: Once a DOI is detected, it queries the Crossref REST API to retrieve the title of the corresponding scholarly work.
Enhancing the Link: The title is then injected as a screen-reader–only link. This means that when a screen reader user navigates to the DOI, they hear the title of the paper rather than the opaque URL.
Maintaining Visual Integrity: For sighted users, the original DOI link remains unchanged—visible, clickable, and easy to copy-and-paste.
Highlighting for Testing: An optional feature highlights updated links, making it easier for developers and testers to see the changes in action.

Get Involved

This project is very much a community effort. The extension is open-source, and we welcome feedback and contributions via our GitLab repository, email, or Community Forum. Your real-life experiences and insights will drive future improvements, ensuring that our solution meets the diverse needs of all users.

Try It Out

If you’re using Firefox, head over to the Firefox Add-on Store and install the DOI Accessibility Enhancer today. If you’re a Chrome user, you can find the extension directly in the Chrome Web Store. If you use a screen reader you’ll experience the difference firsthand - and if you don’t, give it a try with VoiceOver enabled (Command-F5 on a Mac).

Together, we can advance scholarly accessibility and ensure that critical research remains discoverable for everyone.

A Request for Comment - Automatic Digital Preservation and Self-Healing DOIs

Martin Eve — Thu, 29 Jun 2023 00:00:00 +0000

Digital preservation is crucial to the “persistence” of persistent identifiers. Without a reliable archival solution, if a Crossref member ceases operations or there is a technical disaster, the identifier will no longer resolve. This is why the Crossref member terms insist that publishers make best efforts to ensure deposit in a reputable archive service. This means that, if there is a system failure, the DOI will continue to resolve and the content will remain accessible. This is how we protect the integrity of the scholarly record.

I will write another post, soon, on the reality of preservation of items with a Crossref DOI, but recent work in the Labs team has determined that we have a situation of drastic under-preservation of much scholarly material that has been assigned a persistent identifier. In particular, content from our smaller Crossref members, with limited financial resources, is often precariously preserved. Further, DOI URLs are not always updated, even when, for instance, the underlying domain has been registered by a different third party. This results in DOIs pointing to new, hijacked, and elapsed content that does not reflect the metadata that we hold.

We (Geoffrey) have (has) long-harboured ambitions to build a system that would allow for automatic deposit into an archive and then to present access options to the resolving user. This would ensure that all Crossref content had at least one archival solution backing it and greatly contribute to the improved persistent resolvability of our DOIs. We refer to this, internally, as “Project Op Cit”. And we’re now in a position to begin building it.

However, we need to get this right from the design phase out. We need input from librarians working in the digital preservation space. We need input from members on whether they would use such a service. We are not digital preservation experts and we are acutely aware that we need the expertise of those who are, particularly where we’ve had to take some shortcuts. For instance: we are aware that the Internet Archive is perhaps not the first choice of many digital preservation librarians and specialists, who opt for specific scholarly-communications solutions. However, it is easy, open, and free. Hence, we propose for the prototype to use IA, on the assumption that this will be a proof-of-concept only, which we will expand to other archives if there is demand and once it works.

So: please do read the below and add your comments and questions to this thread in the community forum (link below), or send me queries/concerns by email. It would be excellent if we could receive comments by mid-August 2023. If you would rather comment on a Google doc, that’s also possible.

If enough people are interested, we could also host a community call to discuss this design and its prototyping. Do please, when emailing, let me know if this is of interest.

Project Op Cit (Self-Healing DOIs)

Request for Comment

This document sets out the problem statement, a proposed prototype solution, and a transition path to production if successful.

Proposed Prototype Solution

For members who opt-in to the service, We have a special class of DOI (only for open-access content) where, when the DOI is registered:

We immediately make an archive of the item with any archiving services that care to participate in the project (minimally, the Internet Archive, which is the easiest for us to begin with, but a modular/pluggable archival system). The Internet Archive Python Library should let us submit to them. We could pursue other arrangements with CLOCKSS, LOCKSS, and Portico.
We update the XML to reflect the archives to which it has been submitted.
The DOI landing page is redirected to an interstitial page that we control. This page gives the user access options.
We develop processes to determine whether the original URL “works”. The heuristics that define whether a resource has changed substantially or works need long-term consideration and real-world testing. Using the interstitial page approach will allow us to refine this, with a long-term goal of eradicating it.

Figure 1: The Deposit Process

Figure 2: The Resolution Process

Potential Challenges

Content drift. It would be extremely difficult to detect content change vs. (eg) page structure change, except in the case of binary fulltext. However, we can poll for the DOI at an HTML endpoint and detect when binary fulltext items, such as a PDF, change.
Latency on resolver if lookup is real-time. For this reason, we need a periodic crawler so that resolvers do not wait for real-time detection on access.
If using Internet Archive, the domain owner (at the present moment) can request the removal of content. We would need the capacity to “lock” records that are being used as Op Cit redirection archival copies. This requires a further conversation with the Internet Archive.

Prototype Components/Architecture

Registration Proxy and Database (“Fleming”)

The registration proxy implements a pass-through to the deposit API and hosts a relational database of self-healing DOIs (Postgres). It will be hosted at api.labs.crossref.org/deposit/opcit and clients will have to use this endpoint to deposit. Simultaneously, the proxy will:

Determine the license status of the incoming item.
If the license is open and fulltext is provided, deposit a copy in selected digital preservation archives. Store proof of licensing attestation.
In the case of binary files (fulltext PDF), store a hash of the content.
Store the DOI, binary hash, and all URLs in a relational database under “pending” state.
Pass through the request to Crossref’s content registration system.
Monitor the result of this request and remove stored data if registration fails.
Re-registration through Fleming will update existing entries and re-fix their data against content drift at this time.

Spider (“Shelob”)

A series of components that:

Check that “pending” DOIs have been successfully registered. Remove those that have not and move those that have to “active” state.
Dereference “active” DOIs and ensure that we have the most current URL in case updates have gone directly to the live resolver.
Periodically crawl URLs in the self-healing database.
On HTTP 301 code, update database entry to point to new permanent URL.
On HTTP 302 code, follow the temporary redirect expecting the original content.
On HTTP 4xx codes, mark the entry as dead.
On HTTP 200 code of HTML landing page, parse the page for the presence of the DOI. If the DOI is not present, mark the entry as dead.

Resolver Proxy (“Hippocrates”)

Display an interstitial landing page with archival versions and an explanation.
At some future point, for active entries, resolve to the stored URL (faster but could be de-synced) or pass the request to the live resolver (requires an extra hop but will always be in-sync with deposit).

Observability and statistics

Metrics we will collect:

Count of DOIs using Op Cit
Count of visitors arriving on Op Cit landing pages
Usage count of each outgoing link/access option

A daily report will present:

Newly “failed” entries that we believe have died
These will be checked extensively, particularly at first, to ascertain whether our failure heuristics are valid
Entries that have recovered

Errors will be logged and monitored via Grafana.

Documentation and Automated Tests

Core assumptions and new behaviours of the platform will be documented as part of the prototype.
Automated tests will be written, especially for the spider (“Shelob”), which must handle a diverse variety of real-world situations.

Prototype Architecture Requirements

Postgres RDS for resolution/self-healing DOI data (AWS).
FastAPI hosting for passthrough proxy (fly.io).
EC2 hosting for the spider (AWS).
FastAPI hosting for resolver proxy (fly.io).

Transition to Production

If this prototype garners popular appeal, a transition to production would need to keep some prototype components and rewrite others.

“Fleming” would need to be rewritten as a deposit module / integrated with Manifold’s (the next-generation system at Crossref) deposit. If this would create too much overhead, it need not be a blocking process in the deposit.
“Shelob” would continue to need to run continuously and to scale with the adoption of self-healing DOIs unless one of the other options were used.
Prototype architecture will be written so that spidering can be distributed between several servers, if required.
“Hippocrates” would need to be integrated into the live link resolver. Depending on how a field for a self-healing DOI is embedded in Manifold, this may not need any additional database hits.

Back Content

We also have a database of back content stored by the Internet Archive, mapped to DOIs where they have been able to do so. This data source could be used to enable self-healing DOIs on all content in this archive.

Crossref Research and Development: Releasing our Tools from the Ground Up

Martin Eve — Wed, 21 Jun 2023 00:00:00 +0000

This is the first post in a series designed to showcase what we do in the Crossref R&D group, also known as Crossref Labs, which over the last few years has been strengthened, first with Dominika Tkaczyk and Esha Datta, last year with part of Paul Davis’s time, and more recently, yours truly. Research and development are, obviously, crucial for any organisation that doesn’t want to stand still. The R&D group builds prototypes, experimental solutions, and data-mining applications that can help us to understand our member base, in the service of future evolution of the organisation. One of the strategic pillars of Crossref is that we want to contribute to an environment in which the scholarly research community identifies shared problems and co-creates solutions for broad benefit. We do this in all teams through research and engagement with our expanding community.

For example, if the metadata team wants to implement a new field in our schema, it helps to have a prototype to show to members. The Labs team would implement such a prototype. If we want to know the answer to a question about the 150m or so metadata records we have – e.g. how many DOIs are duplicates? – it’s the Labs team that will work on this.

When building such prototypes, which can often seem esoteric and one-off, though, it can be easy to believe that there is no way anybody else would re-use our components. At the same time, we find ourselves consistently working with the same infrastructures, re-using different code blocks across many applications. One of the tasks I have been working on is to extract these duplicated functions and to get them into external code libraries.

Why is this important? As many readers doubtless know, Crossref is committed to The Principles of Open Scholarly Infrastructure. For reasons of insurance, everything we do and newly develop is open source and we want our members to be able to re-use the software that we create. It’s also important because, if we centralize these low-level building blocks, we make it much easier to fix bugs when they occur, which would otherwise be distributed across all of our projects.

As a result, Crossref Labs has a series of small code libraries that we have released for various service interactions. We often find ourselves needing to interact with AWS services. Indeed, Crossref’s live systems are in the process of transitioning to running in the cloud, rather than our own data centre. It makes sense, therefore, for prototype Labs systems to run on this infrastructure, too. However, the boto3 library is not terribly Pythonic. As a result, many of our low-level tools interact with AWS. These include:

CLAWS: the Crossref Labs Amazon Web Services toolkit. The CLAWS library gives speedy and Pythonic access to functions that we use again and again. This includes downloading files from and pushing data to S3 buckets (often in parallel/asynchronously), fetching secrets from AWS Secrets Manager, generating pre-signed URLs, and more.
Longsight: A range of common logging functions for the observability of Python AWS cloud applications. Less mature than CLAWS, this is the starting point for observability across Labs applications. It supports running in AWS Lambda function contexts or pushing your logs to AWS Cloudwatch from anywhere else. It also supports logging metrics in structured forms. Crucially, the logs are all converted into machine-readable JSON format. This allows us to export the metrics into Grafana dashboards to visualize failure and performance.
Distrunner: decentralized data processing on AWS services. Easily the least mature and experimental of these libraries, distrunner is one of the ways that we distribute the workloads of our recurrent data processing. A number of the Labs projects require us to run recurrent data-processing tasks. For instance, my colleague Dominika Tkaczyk has developed the sampling framework that is regenerated once per week. We use Apache Airflow (and, specifically, Amazon Managed Workflows for Apache Airflow) to host these periodic tasks. This is useful because it gives us quick, visual oversight if tasks fail. However, the Airflow worker instances on AWS are quite severely underpowered and unsuitable for large in-memory activities. Hence, the sampling framework fires up a Spark instance for its processing. Often, though, we do not need the parallelization of Spark and just want to be able to run a generic Python script in a more powerful environment. That’s what distrunner is designed to do. The current version uses Coiled but this may change in the future.

While these tools will be useful to nobody except programmers – and this has been quite a technical post – there is a broader philosophical point to be made about this approach, in which everything is available for re-use, “from the ground up”. The point is: we also try, in Labs and in the process of “R&Ding”, to work without privileged access. That is: I don’t get “inside” access to a database that isn’t accessible to external users. I have to work with the same APIs and systems as would an end-user of our services. This means that, when we develop internal libraries, it’s worth releasing them. Because they use systems that are accessible to any of our users.

I should also say that our openness is more than unidirectional. While we are putting a lot of effort into ensuring that everything new we put out is openly accessible, we are also open to contributions coming in. If we’ve built something and you make changes or improve it, please do get in touch or submit a pull request. Openness has to work both ways if projects are truly to be used by the community.

Future posts – coming soon! – will introduce some of the technologies and projects that we have been building atop this infrastructure. This includes a Labs API system; new functionality to retrieve unpaginated datasets of whole API routes; a study of the preservation status of DOI-assigned content; and a mechanism for modeling new metadata fields.