Test failure is due to connection issue, tests pass.

What is still missing/not working properly regarding the logging (potentially new issues to be created):

1. The Processor Servers cannot log to a file (under /tmp/ocrd_server_{processor_name}.{pid}.log) both their and the output of the specific OCR-D Processor is missing. Note that the Processing Server which has similar configurations to the Processor Server can in fact log its output to a file (under /tmp/ocrd_processing_server.log). The separate log files for the caching mechanisms are also fine (under /tmp/ocrd_processing_server_cache_locked_pages.log and /tmp/ocrd_processing_server_cache_processing_requests.log).
2. The Processing Workers can log their output (under /tmp/ocrd_worker_{processor_name}.{pid}.log) regarding the RabbitMQ message consumption and final job status of the triggered OCR-D processor. This works as expected considering that the output of the specific OCR-D processor should not go inside that log.
3. The Deployer module, when deploying the Processing Workers redirects the deployed process/es stdout+stderr matching the processor_name to the /tmp/deployed_{processor_name}.log file. That log file contains both the Processing Workers output and the separate processing steps of OCR-D processors output.

Suggestions:

• It would be easier to wrap our heads around if the run_processor and run_cli methods provided a way to redirect their stdout/stderr to a file path.
• The run_cli method currently triggers a subprocess but only returns the return code of the finished process. If the run_cli supports an extra parameter that accepts a file path, and if set, writes the stdout+stderr of the finished process to that file path - that would be great. At least it is the simplest approach I have in mind currently. Then the Processing Worker could just pass the correct logging file path based on the processing job_id.
• The run_processor method is a bit different. Since no subprocess is started there, potentially we would have to find a way to get the stdout+stderr of the Pythonic processors. This again could happen by supporting an extra file path parameter, and if set, that would create a logging file handler based on the provided path and duplicate the logging to that file.

• It would be easier to wrap our heads around if the run_processor and run_cli methods provided a way to redirect their stdout/stderr to a file path.

For run_cli, it is easy to do and makes sense.

For run_processor, it's more complicated. Since these are called in the same process, redirecting STDOUT7STDERR is tricky. It is certainly possible to mess with sys.stdout but there is a mechanism specifically for this - logging.

How about we add a processor-wide log attribute that holds the logger the processor uses in process, i.e. have

def __init__(...):
  self.log = getLogger('ocrd.ocrd-tesserocr-recognize')

Once we have instantiated the processor, we can add a logging.FileHandler to processor.log which points to a provided file path.

Once we have instantiated the processor, we can add a logging.FileHandler to processor.log which points to a provided file path.

Not sure if that would be enough for what we are trying to achieve. Sure adding a logging.FileHandler will capture everything related to that logger but there will be no differentiation among multiple initializations of that specific processor. 3 Processing Workers initializing the same OCR-D processor will end up logging to the same file since the logger name will be the same. This will not be the case if the logging.FileHandler is initialized inside the run_processor method. However, even then we still do not fully get what we are trying to achieve. Attaching the file handler to that specific logger will omit other relevant logging output from other loggers say ocrd.workspace. Is that what we want to achieve though?

After #1109, now it is clear how we can achieve processing job level file logging.