Some searches are routinely getting an “Internal Server Error” in response. The problem has to do with the way Crossref Metadata Search retrieves data from our API.
As an initial fix, we’re going to make a change to Crossref Metadata Search this coming Monday, March 7th. We will be removing the facets that currently appear on the left-hand side of the results page. These are used to limit search results, for example, by content type, publication year, publisher, etc.
The facets are very resource-intensive, and removing them should largely prevent searches from timing out and getting the “Internal Server Error” responses. We believe this will be a big improvement in the short term, even if it does limit certain ways of searching.
After that change, we’ll evaluate the performance of Crossref Metadata Search and consider how best to bring back facets over time, while maintaining the functionality of the tool.
If you have any thoughts, questions, or feedback, please reply here to let us know.
I understand your reasons, but the lack of any way to filter results has made your database not useful for what i need. i want to use it to add references to my ORCID number, which I just created. I’ve been publishing for years, but never used an ORCID number before. But when I search on my name I get 99,078 results. it seems to find any paper with “Neisser” in it and ignore that I also typed my first name.This is impossible for me to use. I couldn’t find any mention of using Boolean operators or anything to make a more narrow search in your help link. Is there any way to do better now that facets have been removed?
Unfortunately, our API does not support Boolean or exact string searching at this time. The same is true for our Metadata Search interface (search.crossref.org), because it uses an API query against the full metadata records (not just the author names).
This is something we get asked for a lot, and I can note your request to our API product manager to take into account when prioritizing future improvements.
So, unfortunately, there’s no way to limit the search results to just those items where both your first and last name are included in the item’s author metadata.
Instead the results are ranked in order of their relevancy score, which takes into account the words that are queried and their prevalence in the item’s metadata and their proximity to one another. Most of the time this means that the most relevant results will appear first in the list and as you scroll down to later results, you’re more likely to encounter results that are not what you’re looking for. As things are currently structured, each user has to decide where the cutoff point is for their particular search.
It’s very nice to get personal replies to my question. Thank you.
I did discover that if I only search only on my last name instead of my first and last name, I get about 1100 results instead of about 90000. About a hundred of those results should be my articles. But your database is very extensive and my father, my grandfather, my great uncle and my brothers, who all share my name are judged more important in its relevance logic. So one day when I have time, i may do 110 page downs and look for my articles.
I have another question if you don’t mind. I found that if I searched on just “Neisser” instead of “Mark Neisser”, the search came down to 1100 hits or so, so I thought I would look through these. However, once I got to page 10, I couldn’t advance any further. That is, it seems like I am only allowed to see 100 of the search results and nothing more. Is there any way to fix this?
This ten-page limit in our search interface is by design to discourage users from using our search like an API, since it is not intended to function as an API. Additionally, if our users are anticipating paging through more than ten pages of results, we typically ask them to use our REST API anyway - which, for others who are reading, is available at api.crossref.org.
Generally speaking, in this situation, I recommend increasing your search terms instead of decreasing them. That’s going to return more results, but the order of those results will be returned to you in order of relevance. For instance, if you know that the definitive title of one of your works is ‘Searching for Publications’ you might include that in your search, like this: Crossref Metadata Search
Thanks for trying this. I actually don’t know what an API is or what it is your picture shows and I have no idea what you mean about discouraging use of your service as an API. I am not sure what your database is actually used for since it’s so hard to search it. You suggested using your “REST API”, but I looked at the link and honestly, it’s gobbledygook to me. I have no idea what most of the paragraphs are talking about. Can you please suggest something more understandable that I can use to link references to ORCID?
By the way, I think you didn’t find anything about me in your search because you spelled my name “nessier” in the search instead of “Neisser”. I do have a lot of publications and a lot are in your database. I found some in the top ten pages of my search and I am sure there are more. Since I come from a family of academics, there are more Neissers than me and you algorithm thinks they are important so some of them come at the top of the search results.
You’re absolutely correct; there are many other Neissers in these results. I see 1116 total results for ‘Neisser’.
As you may know, our search interface, and then the process of adding your works to your ORCID iD, will be much more straightforward if you have the DOIs associated with your works. Here are a few that I have found that include Neisser in the bibliographic metadata and also include Mark in the author name field:
You should be able to click to add this work to your ORCID record using the link I have circled in the result.
Thus, with this list of DOIs, you can easily add these to your ORCID record.
Here is a spreadsheet that contains all records with the word ‘Neisser’ in the bibliographic metadata. If you search for the word ‘Mark’ within this spreadsheet, you should be able to populate a complete list of DOIs registered with your name: user_url_query_2023-04-24-12-51.csv (750.6 KB)
The spreadsheet is great. I started publishing before the advent of doi’s and so I don’t know the one for all my papers. This will make it possible for me to add them to ORCID. Thanks very much.
