When AwareIM dies

kklosson

I continue to have issues of my server simply dying - it just stops responding to anything. The memory usage of the two java processes is fine, cpu consumption is fine, there is really no reason for the server to stop responding but is does. I have to restart the AwareIM stack then all is well. I cannot believe that the system is overwhelmed - can't me more that 20-30 users. Running build 2028, 64 bit JDK, latest MySQL connector. Very frustrated...

aware_support

You need to capture the output of Tomcat and Aware IM Server when "death" happens - it can be million of reasons not related to Aware IM. We do it by running Aware IM as a service (albeit in 32-bit), in which case all the output is in the logs.

kklosson

To be clear, you are referring to the Tomcat and AwareIM logs available at the AwareIM Control Panel, yes?

A. I'm afraid I don't understand the significance of running AwareIM as a server relevant to the logs.
B. I normally examine both logs when the stoppage occurs and normally don't see anything that appears to be a problem. A recent stoppage produced the attached in the Tomcat log that was abnormal....

aware_support

The log and service is relevant because often you are not present when something happens so you cannot capture the output. You have to be present right at the moment when the server dies, otherwise output may become overwritten with stuff that happens after the problem. With the logs captured in a file you can look at the entire history - especially if you roughly know the time of the problem.

kklosson

So I have a situation where I cannot login nor can some others. But as I look at the logs there is considerable activity occurring. It seems that something has reached its max capacity. Plenty of RAM and CPU overhead available. Nothing in either the AwareIM or Tomcat logs that say anything useful. Then things slow down, less log activity, and I assume the system has simply died. Restart AwareIM control panel to restore operations. There's almost no question that I am failing under load of users but can't figure out what to change to make it better.

I have:
Build 2028
Windows server 2008 R2
AWS EC2 Intel Xeon CPU ES-2650 @ 2.00GH
8GB RAM
2048 on AwareIM
2048 on Tomcat
64 bit JDK
Latest MySQL connector
Stock Tomcat installation

Rennur

Right now I have 25 users logged in and the memory AwareIM server is consuming is just under 2,600,000k.
System usage is already above your allocated settings of 2048 with only 25 users. My BSV is 3.3mb.

No performance issues but it may be due to a low number of logged in users.

Rennur

Have you cons

Have you considered Tomcat 64 bit and replacing the activeMQ with the latest version.

As per support, ActiveMQ processes everything so it would be a good idea to be running with the latest version.

kklosson

First time I have heard of it. Will study.
Thanks.

customaware

Is there anything in Context History table Kingsley?

If so..... delete all those records and try again.

I had a Process that was behaving badly and it filled that table up to the point where nothing would work.

Support found the problem and removing the Context History records fixed it.

kklosson

I have a tabled called execution_contexts but it's empty.

kklosson

Interesting...

I believe the problem I have been having just lately is that the Tomcat SessionTimeout (AwareIM\Tomcat\webapps\AwareIM\webapps.props) is normally set to 7200000 (120 minutes). Somehow, not by my doing, this value was reset to 7200. Aside from be way too short, it is in conflict with the sessiontimeout setting in BASServer.props. I can say that I have seen this before, where the timeout value in webapps.props gets mysteriously altered. I have seen it changed to ridiculous numbers like -203409834054, which caused all kinds of problems. So I must assert that there is some action in AwareIM (or Tomcat maybe) somewhere that will change this value autonomously. Things are better now.

customaware

Thanx for posting it Kingsley.
Good one to keep in mind.

kklosson

Although even with that issue repaired, I am still experiencing instances of the system becoming unresponsive.

customaware

Have Support not been able to shed any light on the problem Kingsley?

kklosson

There is not a lot to go on. Support says that reviewing the logs is critical, and I do, but I see nothing in there that is informative. I can see that the server is still processing requests though I am unable to login, or if I am logged in the system becomes totally unresponsive. I really don't know if the load is doing this or what. Just clueless at this point.

customaware

Not that I have any clues but where is this on AWS and what
is the configuration?

kklosson

in Oregon, it's m1.large.

customaware

Windows, MySQL on RDS ...right?

RAM? SSD?

I am sure it is not an issue but should as anyway..... how much disk space is free on the application server?
I did have a situation with similar traits to yours when I realised that I was running out of hard disk space.

kklosson

I'm not on RDS but there are no capacity or performance limitations.

RLJB

Kingsley

Here's a left of field guess... Post, or pm me, your MySQL my.conf file pls. You know we suffer similar on one of our prod servers, and only one (not all), but I have theory it's a MySQL setting for us.

« Previous Page Next Page »