The "problem", such as it is, is on the server side: the client has made a well formed request, but the server can not satisfy it. So I'm inclined to a "Server Error", 5xx status code.
Quoth RFC 7231 (the current HTTP standard, emphasis added):
The 5xx (Server Error) class of status code indicates that the server
is aware that it has erred or is incapable of performing the
requested method. Except when responding to a HEAD request, the
server SHOULD send a representation containing an explanation of the
error situation, and whether it is a temporary or permanent condition.
Note
- "erred or is incapable of performing the request": despite their title of "Server Error", they are not just for server errors.
- "temporary or permanent": these codes are suitable for temporarily unavailable resources, like yours.
Of the available codes, I'd say 503, "Service Unavailable" was the best fit:
The 503 (Service Unavailable) status code indicates that the server
is currently unable to handle the request due to a temporary overload
or scheduled maintenance, which will likely be alleviated after some
delay. The server MAY send a Retry-After header field... to suggest an appropriate amount of time for the
client to wait before retrying the request.
Note:
- "likely be alleviated after some delay": true for your case.
- "temporary overload": not pedantically true for your case. But, it could be argued, were your server much faster, the batch processing would have already been done when the client made the request, so it is a kind of "overload": the client is asking for resources faster than the server can make them available.
- Retrying is suitable for your service, so your reply ought to include a
Retry-After
value. You could provide as the value the estimated completion time of the next execution of the batch process, or the execution interval of the batch process.
Defining your own 5xx status code (591, for example), although permitted, would have the wrong semantics:
a client MUST
understand the class of any status code, as indicated by the first
digit, and treat an unrecognized status code as being equivalent to
the x00 status code of that class
Clients would treat your own status code as 500, "Internal Server Error", which would not be right.
204
"No Content"? Is indicates that the server has successfully processed the request and is not returning any content [at this time].