Hello,
I have an ActiveMQ instance that has been running nicely for a couple of months, but doesn't seem to clean up its KahaDB logs like I expect it to. (More specifically, I expect to have like 1 log file, but in reality I have ~1500 log files with 49G of data.)
ActiveMQ has processed about 12M messages over the past couple of months, and has roughly 106K messages currently pending (in DLQs):
[cid:8627D172-A47C-4687-BDE1-89CFA12B5F38]
Given the number of messages currently pending, I would not expect the KahaDB logs to be very big. In reality, they're about 49G:
$ cd apache-activemq-5.8.0/data/kahadb/
$ ls -lht | head
total 49G
-rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
-rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
-rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
Based on the maximum log ID, almost none of the log files appear to have been cleaned up:
$ ls db-*.log | wc
1540 1540 17421
I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla configuration is Java heap size, which shouldn't affect whether or not DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with Camel. Obviously from the number of queues, I have several different workflows in flight, but here's a representative example of how I'm using ActiveMQ in my Java code:
from("activemq:statuslinks")
.bean(new LinkExtract())
.split(body(String.class).tokenize("\n"))
.filter(new LinkHandled(jdbcConnectionString))
.to("activemq:linkunwind");
The only problems I'm seeing in my activemq.log file appear to be "normal." I get rashes of transport exceptions every few weeks, which I have associated (rightly or wrongly) with hitting the web admin interface. Given the infrequency and nature of the errors, I don't think they explain why the logs wouldn't be deleted. In case it's useful, though, here are a couple of representative lines from the log file:
2013-12-13 09:46:52,269 | WARN | Transport Connection to: tcp://127.0.0.1:39604 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://127.0.0.1:39604 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker
2013-12-13 14:43:41,215 | WARN | Transport Connection to: tcp://127.0.0.1:35711 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:35711 [ at ] 61616
Given the use case — very simple message processing using Camel — and ActiveMQ's vanilla configuration, I would expect the log files to be tiny. Clearly, though they're not. Does anyone have any idea why those files aren't getting deleted? I'm sure I'm doing (or not doing) something very silly, but after pouring over the logs and documentation, I just can't figure out what I'm doing wrong. Does anyone have an idea why my logs wouldn't be getting cleaned up?
Thanks everyone in advance for your attention and help! And thank you ActiveMQ devs for such a wonderful product! :)
Andy Boothe
I have an ActiveMQ instance that has been running nicely for a couple of months, but doesn't seem to clean up its KahaDB logs like I expect it to. (More specifically, I expect to have like 1 log file, but in reality I have ~1500 log files with 49G of data.)
ActiveMQ has processed about 12M messages over the past couple of months, and has roughly 106K messages currently pending (in DLQs):
[cid:8627D172-A47C-4687-BDE1-89CFA12B5F38]
Given the number of messages currently pending, I would not expect the KahaDB logs to be very big. In reality, they're about 49G:
$ cd apache-activemq-5.8.0/data/kahadb/
$ ls -lht | head
total 49G
-rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
-rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
-rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
Based on the maximum log ID, almost none of the log files appear to have been cleaned up:
$ ls db-*.log | wc
1540 1540 17421
I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla configuration is Java heap size, which shouldn't affect whether or not DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with Camel. Obviously from the number of queues, I have several different workflows in flight, but here's a representative example of how I'm using ActiveMQ in my Java code:
from("activemq:statuslinks")
.bean(new LinkExtract())
.split(body(String.class).tokenize("\n"))
.filter(new LinkHandled(jdbcConnectionString))
.to("activemq:linkunwind");
The only problems I'm seeing in my activemq.log file appear to be "normal." I get rashes of transport exceptions every few weeks, which I have associated (rightly or wrongly) with hitting the web admin interface. Given the infrequency and nature of the errors, I don't think they explain why the logs wouldn't be deleted. In case it's useful, though, here are a couple of representative lines from the log file:
2013-12-13 09:46:52,269 | WARN | Transport Connection to: tcp://127.0.0.1:39604 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://127.0.0.1:39604 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker
2013-12-13 14:43:41,215 | WARN | Transport Connection to: tcp://127.0.0.1:35711 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:35711 [ at ] 61616
Given the use case — very simple message processing using Camel — and ActiveMQ's vanilla configuration, I would expect the log files to be tiny. Clearly, though they're not. Does anyone have any idea why those files aren't getting deleted? I'm sure I'm doing (or not doing) something very silly, but after pouring over the logs and documentation, I just can't figure out what I'm doing wrong. Does anyone have an idea why my logs wouldn't be getting cleaned up?
Thanks everyone in advance for your attention and help! And thank you ActiveMQ devs for such a wonderful product! :)
Andy Boothe