There seems to be a problem with the OAI-PMH Plus feed, since September 3rd 2023. We have two daily import that makes requests for articles and books, here’s an example of the requests we make:
Thanks for your message, and welcome to the Community Forum! We’ve heard from a few Metadata Plus subscribers about this.
It’s taken us a little longer than we would have liked to identify the problem, but our customer relationship management system underwent a routine, quarterly release on Sunday around 2000 UTC that introduced this bug. Some of our members’ internal identifiers got out of sync as a result of that release. Unfortunately, it sounds like you might be one of them. Our technical team is working on the fix now.
We deployed a fix to this bug. All of the Plus tokens that I have spot checked are providing access to their Metadata Plus subscribers. Can you let me know if you continue to experience problems on your end?
We’re performing an import on books (set=B) only which is failing.
We also have a similar regular import on articles/journals (set=J) which has been working fine over this same period. Can you please investigate further?
While the J set has more individual DOIs, the B set has many, many more setspecs. So, a timeout might be expected when you’re asking for data for the whole set. The workaround, in that case, would be to break it down by prefix, since you can’t narrow the time window any further.
I’ll check with our technical team to see what the expected behavior is, and file a ticket if they believe that this is a bug.
Any update on this issue? We’re still seeing a lot of timeout issues on the books endpoint, and it’s been ongoing since a few months back.
Prior to this, we’ve never really seen any issues with this endpoint, performing a daily export going at least 3-4 years back. Has something changed in recent months that have caused this?
Just letting you know that the issue has now been resolved, we’ve not seen the timeouts in the past few days since you resolved the wider incident of the 500 timeout errors.
Thanks for confirming that this is still an ongoing issue for you. @ppolischuk our product manager for metadata retrieval has looked into the issue and, unfortunately, is unable to replicate. For Patrick, response times are high, between 30 and 45 seconds in his testing.
Similar requests for B sets from a single day, both with larger and smaller completeListSize values are also working fine, albeit with relatively long response times.
Given all this, we’ve asked Jon Stark on our technical team to take a look for us to see if he can determine what might be causing the 500 status code timeout errors you’re reporting.
Hello @gdfry, I’m glad the issue resolved itself from your perspective. One of our developers spent some time investigating these timeouts and found that they correspond closely with some database issues we were having during the reported period.
We’ll keep an eye out for any further performance issues. Thanks for reporting this to us.