'Performance Issues Saving ActiveMQTextMessages

I am trying to consume ActiveMQTextMessages (received as Message objects) and store into a file given a very high data rate (9K a second is the target). I am running into performance problems trying to save the messages into a set of files.

I am currently receiving at 30K a second and pushing into a ConcurrentLinkedQueue of which I have a configurable set of workers popping off the queue until they reach a max internal queue amount. This all seems fine. The issue I am having is when I am passing the concatenated text (longest running time I think) to the compression algorithm. I would be willing to store the textmessage received from the message but, when I stored it out, what would be a more efficient way to handle the following pseudo?

FileOutputStream fout = new FileOutputStream(op);
Deflater df = new Deflater(Deflater.BEST_SPEED);
DeflaterOutputStream out = new DeflaterOutputStream(fout, df);
while(!localQueue.isEmpty())
{
    Message msg = localQueue.remove();
    ActiveMQTextMessage amqtextmesssage = (ActiveMQTextMessage)msg;
    String textMessage = amqtextmesssage.getText();
    String jmsCorrID = amqtextmesssage.getJMSCorrelationID();
    Destination destinationID = amqtextmesssage.getJMSDestination();
    long jmsTimeStamp = amqtextmesssage.getJMSTimestamp();

   //FORMAT FOR SPECIFIC OUTPUT TO BE POSTPROCESSED
   String result = String.format(/*format strings here*/);
   InputStream is = new ByteArrayInputStream(result.getBytes());
   
   int i;
    while ((i = is.read()) != -1) {
        out.write((byte) i);
        out.flush();
    }
    is.close();
}


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source