Too much logging activity over the weekend led to the following error being thrown by ColdFusion:
Message: No space left on device
StackTrace: java.io.IOException: No space left on device at
java.io.FileOutputStream.writeBytes(Native Method) at
By this morning pages on the ColdFusion web site were not loading at all. The disc (12Gig) was over 99% used up. We moved several files to the second disc and now it's at about 80%, much lower than where it's always been. We're going to direct logging activity to the second disc (100Gig) to prevent a repeat. Having created space on the disc we restarted apache and coldfusion but still pages are not loading.
When we run
top -H it appears that java is running at close to
100% CPU. Does anyone have a clue what's going on or what info I need to provide so someone can figure it out?
The set up is AWS, ubuntu 13.04, coldfusion 10, mysql (rds).
I have made some really strange but hopefully helpful observations. I am still trying to locate a tool to help with getting thread dumps. Whenever I restart ColdFusion, most pages load fine and CPU usage seems normal, mostly 0.7 - 1.5% but once in a while spikes of up to 10% can be seen. But there's one particular page, which when I try to load, causes CPU uses to rise to 97% always. The pages that load fine have a simple query reading data from one table. This problematic page has an inner join, and reads data from two tables. I don't know how helpful this is but I think it is too consistent to be insignificant.
And since this issue started the following errors appeared for the first time and there're hundreds of lines of it: