When is the best time period to check submissions

[mohamad]

After I deposit XML to the crossref via POST API, immediately check the download of the submission ( Use HTTPS to retrieve logs).
but I give this response

<?xml version="1.0" encoding="UTF-8"?>
<doi_batch_diagnostic status="unknown_submission">
  <submission_id>0</submission_id>
  <batch_id />
</doi_batch_diagnostic>

When is the best time period to check submissions?

[ifarley]

Hi @mohamad ,

Thanks for your question. Generally speaking, most submissions are processed within a few (say, five to ten) minutes of receipt. That said, there are a number of variables to consider:

1. the size of your submission - XML including a single DOI with no references in the metadata record sent to us typically takes a few minutes to process; XML including many DOIs that include reference metadata (since we’re processing those references and establishing links between the DOIs being registered and the DOIs that those newly-registered DOIs cite) would take longer

2. the number of submissions you’re sending us simultaneously - if you’re sending us thousands of files for us to process, that will take us time to process; we have the capacity to run 30 concurrent threads in our submission queue and we’ll process five submissions per username concurrently (barring over-all demand on those threads) - we have logic built into the system that makes processing equitable

3. the number of submissions in the queue - if we have many submissions from a number of members, that’s going to take us longer to process than if we only have a few submissions from a handful of members
   

   

At the time of this writing, we have 26 threads processing submissions concurrently and 1443 submissions pending in the submission queue. This view in the submission queue is open and transparent, so when you log into our admin tool - doi.crossref.org - you can see all submissions processing and pending for ALL Crossref members.

4. Maintenance - we pause the processing of submissions into the queue on Tuesdays between 1400 and 1700 UTC to fix bugs and release new features; during that time, we accept submissions, but it does take extra time on Tuesdays (when we have a maintenance release) for us to process all submissions because of that pause in processing. You can monitor the status of our system, including those maintenance windows here: https://0-status-crossref-org.libus.csd.mu.edu/

Here’s an example of a recent maintenance release (and the updates we post to the status page):

5. Unexpected incidents (problems/bugs) - we had one of these at the end of last week; we can’t predict them, but they can and do cause submission processing delays. I’m not sure about the specifics of your example, but based on the timing, this could be what happened with your submission. We announce all of these incidents on our status page. Here’s the most-recent incident: Crossref Status - Possible problem with Crossref content submission - now solved

So, again, generally speaking, most submissions take us a few minutes to process. But, there are several factors that can extend the amount of time it takes us to process a member’s submission to us. In some cases, like on a Tuesday during our maintenance window, the timing of our processing can extend to hours. In other, rare cases, like the incident from the end of last week, the timing of our processing can extend to days.

Can you provide me with your DOI prefix and/or Crossref username so that I can provide you with specifics about the processing of your submissions?

Thanks in advance,
Isaac

[mohamad]

Your explanation was very complete.

Actually, We are developing an automated system that saves metadata to crossref and receives from it. Finally, we want to check queued items periodically (now 30 minutes).

[ifarley]

I would expect that a periodic check running every 30 minutes would catch most items registered by your username, but there will be times where the content will not yet have been processed (e.g., Tuesdays when we have a release).

-Isaac