I've been testing the AsyncJsPublisher with data volumes in excess of 100,000 messages a second and it's considerably slower than my own code which uses a bounded ack queue.
I'm seeing average ack latencies in excess of 1 second in some instances whereas my own code is averaging around 4 milliseconds. In both instances my max in-flight queue size is 10,000.
I suggest updating the publishers internals to reject new messages or accept some kind of timeout and if we've exceeded the max in-flight limit, then let the client know they need to slow down.
Right now, I'm pushing more data than the publisher can handle and it's causing a build up in memory pressure and huge thread contention problems.
I've been testing the AsyncJsPublisher with data volumes in excess of 100,000 messages a second and it's considerably slower than my own code which uses a bounded ack queue.
I'm seeing average ack latencies in excess of 1 second in some instances whereas my own code is averaging around 4 milliseconds. In both instances my max in-flight queue size is 10,000.
I suggest updating the publishers internals to reject new messages or accept some kind of timeout and if we've exceeded the max in-flight limit, then let the client know they need to slow down.
Right now, I'm pushing more data than the publisher can handle and it's causing a build up in memory pressure and huge thread contention problems.