CPG speedup

[siemdejong]

The CPG algorithm in this repo is much much slower than fastinference (minutes instead of seconds). Possible reasons after inspecting the original tissue_seg repo

1. (4x) patch size. Segmenteer uses 224 because the onnx export needs that, but I can re-export and make it dynamic. The original uses 512, which could provide a 4 times speedup maybe.
2. (5x) number of workers: segmenteer now has 1 by default, because apparently there are issues with openslide on mac. This needs to be investigated by someone with a mac. The comment that suggests this issue might be AI-generated. fastinference uses 5 readers and 5 writers simultaneously. We can increase the number of workers that trident uses
3. (2x) spacing: Segmenteer uses 4x (2.5mpp), the original uses 4 mpp.
4. (9x) batch_size: original is 70, we have 8 by default.

Calculation on envelope: all in all, I expect roughly a speedup of 4x5x2x9=360, which would result in a 529s/360=2s inference runtime(?), which is similar to fastinference!

Other ways to gain speedup here (which I think we can apply because it is our own model):

1. set dataloader pin_memory to True for asynchronous memory copy from host to GPU, but this is external, so we cannot control it
2. export with channels last format: https://docs.pytorch.org/tutorials/intermediate/memory_format_tutorial.html#converting-existing-models and make a transpose .view of the input and output images.
3. export onnx with tensor rt and run with TensorrtExecutionProvider. This does better optimization than the CUDA execution provider.
4. Set batch size to a multiple of 8 to map to tensor core dimensions
5. quantization, my knowledge on this is limited and I'm not sure if data is required for this to work or if one could always do this and expect it to work.
6. post training quantization, but this needs a calibration dataset so this needs some work.