Self-hosted crossref API

Technical Support Engineering
, ,
1

Dear community,

my task to set up a in-house instance of the crossref-API. Several people of our institute already use the public API for enrichment of our data. i would like to set up our own instance in order to start large batch procresses usind existing scripts.

what I have so far:

  • on Gitlab: crossref/rest_api :
    • the example using “user/index-feed” works, i can query my local API.
  • I can upload JSON data from the crossref dump to Elastic and query it on Elastic, but not using the API
  • Can you point me in a direction how I can properly upload the JSON dump for querying using the API?
  • OR: where I can get a dump in UNIXSD Format (which seems to be easy to upload)

best regards,
klaus

2

Hello Klaus, we’re excited to hear about your attempts to run an instance of the Crossref REST API locally. While this isn’t something we formally support, and thus don’t have any documentation to point you to, we put our heads together and have a few bits of feedback that we hope might be helpful.

To run the API connected to Elasticsearch, you can build the code into a docker image and run it locally. It needs some other settings but we may be able to help with the specifics.

The REST API is currently configured to ingest an XML snapshot of our data, which is only available to subscribers of our Metadata Plus service. And unfortunately we don’t have experience loading the JSON-formatted public data file into the Elasticsearch format. We’ve previously set up Elasticsearch with a schema but can’t speak to translating the JSON snapshots to match that schema, or if it’s already in that format since it originated from Elasticsearch anyway.

The easiest way forward is probably to subscribe to Metadata Plus if you are not already a subscriber. I don’t think these answers fill all the gaps you’ve discovered so far. But if you have any other questions, please let us know and we’ll do our best to follow up.

We’re keen to hear more about your experience trying to run the API locally. If heavy users decide to go this route, that may reduce the burden on our API, so we are interested in facilitating this work if time allows.

Thanks for sharing your progress so far.

Take care,
Patrick

3

Hello Patrick,

Thank you very much for the quick and detailed reply. I now have all the information I need to decide which step to take next towards our goal of a local API / local data set

best regards,
Klaus