awareim causing issues with 200 users loggedIn

aware_support

Aware IM DOES close all the connections - this has been tested many times. You can run a simple test to make sure. Just create a BSV with one simple query and keep refreshing it - see if your MySQL connections keep growing.

Aware IM CAN run multiple processes at the same time, otherwise it would be unusable.

My guess is that you have a fundamental problem elsewhere in your app, but auto-refresh exacerbates it.

You also need to run your system as a service and collect a server and Tomcat logs. When the problem occurs you should carefully study this log for any clues. There are hundreds of theoretical reasons for this problem - memory being one of them. Only the log can give you the answer as to what is going on.

mrbdrm

aware_support wrote
Aware IM DOES close all the connections - this has been tested many times. You can run a simple test to make sure. Just create a BSV with one simple query and keep refreshing it - see if your MySQL connections keep growing.

Aware IM CAN run multiple processes at the same time, otherwise it would be unusable.

My guess is that you have a fundamental problem elsewhere in your app, but auto-refresh exacerbates it.

You also need to run your system as a service and collect a server and Tomcat logs. When the problem occurs you should carefully study this log for any clues. There are hundreds of theoretical reasons for this problem - memory being one of them. Only the log can give you the answer as to what is going on.

Aware opening connections with no info is that normal?
Screenshot 2017-10-17 11.38.17.png
Screenshot 2017-10-17 11.38.48.png

the all mighty Execution context crashing mysql work bench
Screenshot 2017-10-17 11.44.49.png

Question: if i have a process to DISPLAY document, does aware put the document data in the execution context table? could this be the reason for the issues?

i found this on tomcat log from the control panel

SEVERE: Servlet.service() for servlet [LogonOp] in context with path [/AwareIM] threw exception [org.apache.catalina.connector.ClientAbortException: java.io.IOException: An established connection was aborted by the software in your host machine] with root cause
java.io.IOException: An established connection was aborted by the software in your host machine

to clarify my system settings was changed to double the memory and i also separate the database on other server, server ram is at 29% usage when this happen. 12GB free on the hard if that have any effect.

aware_support

Aware opening connections with no info is that normal?

What do you mean "no info"? What you have to check is whether the number of connections keeps growing.

Question: if i have a process to DISPLAY document, does aware put the document data in the execution context table? could this be the reason for the issues?

It does put the record in the table, but it shouldn't cause any problems.

EVERE: Servlet.service() for servlet [LogonOp] in context with path [/AwareIM] threw exception [org.apache.catalina.connector.ClientAbortException: java.io.IOException: An established connection was aborted by the software in your host machine] with root cause
java.io.IOException: An established connection was aborted by the software in your host machine

This is probably the reason why the system becomes unresponsive - the connection is broken for some reason, not sure why. Try Googling for a solution.

customaware

Interestingly, I just googled that error and.....

"This problem can be simply solved by closing Eclipse and restarting it. Eclipse sometimes fails to establish a connection with the Emulator, so this can happen in some cases."

Dunno if it is relevant to you problem but got a big green tick for the right solution to that error.

aware_support

I don't think it has anything to do with Eclipse. Most probably the problem is caused by multiple users uploading/downloading large document files at the same time (see a different thread on this). You need to avoid such scenarios in your application -it can easily put the server to its knees.

Also, if you allow uploads of files larger than 64Mb you need to increase memory for Active MQ (there is a separate thread about it). Otherwise, your server will immediately go down and become unresponsive.

mrbdrm

aware_support wrote
I don't think it has anything to do with Eclipse. Most probably the problem is caused by multiple users uploading/downloading large document files at the same time (see a different thread on this). You need to avoid such scenarios in your application -it can easily put the server to its knees.

Also, if you allow uploads of files larger than 64Mb you need to increase memory for Active MQ (there is a separate thread about it). Otherwise, your server will immediately go down and become unresponsive.

i have limited the user to one request for the document every 60 second no double clicks will pass.

strangely with my test on this method and me trying to over load the server with double clicks i noticed the connections to the database spike +30 BUT they are all closed after a few seconds witch is great 🙂
tomorrow we shall see the real test and if that will resolve the issue.
we don't upload large files thankfully 🙂

BenHayat

mrbdrm wrote
aware_support wrote
I don't think it has anything to do with Eclipse. Most probably the problem is caused by multiple users uploading/downloading large document files at the same time (see a different thread on this). You need to avoid such scenarios in your application -it can easily put the server to its knees.

Also, if you allow uploads of files larger than 64Mb you need to increase memory for Active MQ (there is a separate thread about it). Otherwise, your server will immediately go down and become unresponsive.

i have limited the user to one request for the document every 60 second no double clicks will pass.

strangely with my test on this method and me trying to over load the server with double clicks i noticed the connections to the database spike +30 BUT they are all closed after a few seconds witch is great 🙂
tomorrow we shall see the real test and if that will resolve the issue.
we don't upload large files thankfully 🙂

Are you using AWS version of MySQL they offer or are you running a copy of MySQL on your own instance?
Is it possible AWS takes too long or hangs on to connections?

mrbdrm

if that was the case then why its closing properly after i limit the request to download?

mrbdrm

problem still persists. 🙁

too many connections with no activity on them, (266 connection) and then aware stop responding

the display document fix that i did helped a lot on the size of execution context table but still it reach 300+ rows

Vlad: is there any way i can control the connections to database? any settings or variable to change? i need to know why aware is doing this?

aware_support

You cannot control connections to the database. You need to find out why connections are left open. 200 connections are not that many - it could be a pool of connections. Rather than looking at connections you need to check the log (Aware IM and Tomcat) and see why the server becomes unresponsive.

Note the time of the problem and then see what was happening in the log around that time.

mrbdrm

here is errors i found on tomcat log and aware log when this happen (from control panel):
everything else is normal on the log

Evaluating condition LoggedInRegularUser.EmployIn=OL.ToD OR LoggedInRegularUser.EmployIn=OL.FromD
Getting logged in user...
com.bas.connectionserver.server.D: Internal error. Error reading attribute Unknown column 'EmployIn_DOCDATA' in 'field list'
	at com.bas.connectionserver.server.ConnectionFactory.A(Unknown Source)
	at com.bas.connectionserver.server.ConnectionFactory.sendMessage(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.thin.A.H.A(Unknown Source)
	at com.bas.shared.A.G.A(Unknown Source)
	at com.bas.shared.A.G.E(Unknown Source)
	at com.bas.shared.A.G.D(Unknown Source)
	at com.bas.webapp.thin.A.H.D(Unknown Source)
	at com.bas.webapp.thin.A.B.B(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.handlers.B.C.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.handleRequest(Unknown Source)
	at com.bas.webapp.thin.servlets.WebInterfaceServlet.doPost(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88-)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658-)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Unable to resolve reference attribute
Got XML request <root><query_layout_action query_name='EmailInboxOld' > 
</query_layout_action> 
</root>
com.bas.connectionserver.server.D: Internal error. Error reading attribute Unknown column 'ToD_DOCDATA' in 'field list'
	at com.bas.connectionserver.server.ConnectionFactory.A(Unknown Source)
	at com.bas.connectionserver.server.ConnectionFactory.sendMessage(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.thin.A.H.A(Unknown Source)
	at com.bas.shared.A.G.A(Unknown Source)
	at com.bas.shared.A.G.E(Unknown Source)
	at com.bas.shared.A.G.D(Unknown Source)
	at com.bas.webapp.thin.A.H.D(Unknown Source)
	at com.bas.webapp.thin.A.B.B(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.handlers.B.C.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.handleRequest(Unknown Source)
	at com.bas.webapp.thin.servlets.WebInterfaceServlet.doPost(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88-)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658-)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Unable to resolve reference attribute
WebServlet URL got request with params:
Condition evaluated to true:= OR =
com.bas.connectionserver.server.D: Internal error. Error reading attribute Unknown column 'FromD_DOCDATA' in 'field list'
	at com.bas.connectionserver.server.ConnectionFactory.A(Unknown Source)
	at com.bas.connectionserver.server.ConnectionFactory.sendMessage(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.common.L.A(Unknown Source)
	at com.bas.webapp.thin.A.H.A(Unknown Source)
	at com.bas.shared.A.G.A(Unknown Source)
	at com.bas.shared.A.G.E(Unknown Source)
	at com.bas.shared.A.G.D(Unknown Source)
	at com.bas.webapp.thin.A.H.D(Unknown Source)
	at com.bas.webapp.thin.A.B.B(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.A.B.A(Unknown Source)
	at com.bas.webapp.thin.handlers.B.C.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.handleRequest(Unknown Source)
	at com.bas.webapp.thin.servlets.WebInterfaceServlet.doPost(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88-)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658-)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Unable to resolve reference attribute

Exception in thread "AWT-EventQueue-0" 
java.util.ConcurrentModificationException
	at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966)
	at java.util.LinkedList$ListItr.next(LinkedList.java:888-)

Evaluating condition LoggedInRegularUser.AccessLevel='DepManager'
Condition evaluated to false:RegularEmp=DepManager
??? 18, 2017 11:31:47 ? org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [LogonOp] in context with path [/AwareIM] threw exception [org.apache.catalina.connector.ClientAbortException: java.io.IOException: An established connection was aborted by the software in your host machine] with root cause
java.io.IOException: An established connection was aborted by the software in your host machine
	at sun.nio.ch.SocketDispatcher.write0(Native Method)
	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
	at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:127)
	at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
	at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:173)
	at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
	at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
	at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
	at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
	at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:121)
	at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:257)
	at org.apache.coyote.Response.doWrite(Response.java:492)
	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:384)
	at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:344)
	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:414)
	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:402)
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:97)
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:90)
	at com.bas.webapp.common.WebAppUtils.renderResponse(Unknown Source)
	at com.bas.webapp.thin.servlets.LogonAdminServlet.openStartScreen(Unknown Source)
	at com.bas.webapp.thin.servlets.LogonAdminServlet.doPost2(Unknown Source)
	at com.bas.webapp.thin.servlets.LogonServlet.doPost2(Unknown Source)
	at com.bas.webapp.thin.servlets.LogonServlet.doPost(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88-)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658-)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)

WebServlet URL got request with params:
Got XML request <root><start_process_action process_name="EmailRead" force_no_collection="false" param_values="Email" screen_width="1440" screen_height="769" screen_orientation="landscape" ><object_reference object_name="Email" object_id="290412" /> 
</start_process_action></root>
Calling BAS server
Exception thrown com.bas.connectionserver.server.D ??? ??????? ????? ?????.
com.bas.connectionserver.server.D: ??? ??????? ????? ?????.
Server request completed
	at com.bas.connectionserver.server.ConnectionFactory.A(Unknown Source)
	at com.bas.connectionserver.server.ConnectionFactory.sendMessage(Unknown Source)
	at com.bas.webapp.common.WebAppUtils.sendMessageToExecutionEngine(Unknown Source)
	at com.bas.webapp.thin.A.O.A(Unknown Source)
	at com.bas.webapp.thin.handlers.B.C.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.A(Unknown Source)
	at com.bas.webapp.thin.handlers.XMLRequestHandler.handleRequest(Unknown Source)
	at com.bas.webapp.thin.servlets.WebInterfaceServlet.doPost(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88-)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658-)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Server request completed
Server replied to the original request that some UI operation is required
WebServlet URL got request with params:
Null response returned from the web server

can you make any presumptions passed on this? there is no errors at all when i test the system.
i will try adding timeout on mysql server to close the hanging connections.

Jaymer

wondering what the Windows TASK MANAGER looks like when its hung... w.r.t. Memory & CPU.

mrbdrm

Jaymer wrote
wondering what the Windows TASK MANAGER looks like when its hung... w.r.t. Memory & CPU.

cpu around 2%
memory around 29%

aware_support

This is only a Tomcat log. You haven't attached the server log.

Check out this error message:
SEVERE: Servlet.service() for servlet [LogonOp] in context with path [/AwareIM] threw exception [org.apache.catalina.connector.ClientAbortException: java.io.IOException: An established connection was aborted by the software in your host machine] with root cause
java.io.IOException: An established connection was aborted by the software in your host machine

Your TCP/IP sockets stop working for some reason. The question is whether it's the server that is responsible for this interruption or not. The server log around exactly the same time as the error above should tell. If the server log is normal then you need to Google for a possible cause (this error is thrown by the TCP/IP sockets system, so it is not specific to Aware IM).

If you want us to have a look at the log please purchase a support ticket and send the logs to support@awareim.com

mrbdrm

Thank you Vlad but im not running aware as a service so i don't have full log.

the great news is today is the first day i run a full day without any issues, in fact aware is super fast and impressive.
we have about 10 users in less than 5 minute increment, we reached about 400 concurrent users or less no issues.

my fix was to set a timeout of 5 minutes on MySQL connections.

however i still believe that in high concurrent users heavy load aware will kill it self with MySQL connections. i hope Vlad investigate this more.

BenHayat

mrbdrm wrote
my fix was to set a timeout of 5 minutes on MySQL connections.

What was the default timeout value for MySQL?

mrbdrm

BenHayat wrote
mrbdrm wrote
my fix was to set a timeout of 5 minutes on MySQL connections.

What was the default timeout value for MySQL?

default setting, no timeout

BenHayat

mrbdrm wrote
BenHayat wrote
mrbdrm wrote
my fix was to set a timeout of 5 minutes on MySQL connections.

What was the default timeout value for MySQL?

default setting, no timeout

But as Vlad pointed out, you may have some issues internally with TCP/IP and the longer timeout may seem to hide it but still a problem. It's like you may have some health issue and now you're taking more "Aspirin" to hide the pain.
I still think with help of support you should get to the bottom of this and let the system work on default timeout. It's a good investment of resource to solve this problem.

mrbdrm

BenHayat wrote
But as Vlad pointed out, you may have some issues internally with TCP/IP and the longer timeout may seem to hide it but still a problem. It's like you may have some health issue and now you're taking more "Aspirin" to hide the pain.
I till with help of support you should get to the bottom of this and let it work on default timeout. It's a good investment of resource to solve this problem.

you are right about finding the issue but i can't help with solving it until i switch aware to service and send the log to Vlad.
on the TCP/IP , there is no issue there. nothing has been done to the windows server TCP/IP. if there was one then i should get the error even when i restart aware.
the port is open and there is no limit to the number of connections from windows.
and as bonus Execution context are properly empty when the rush of users are gone. get some records there and they are gone after a few minutes.

one user on the office think the app was moved to local network 🙂 never seen my app this fast.
btw is there a guide on how to run aware as service in this forums? i searched but can't find one.

BenHayat

mrbdrm wrote
BenHayat wrote
btw is there a guide on how to run aware as service in this forums? i searched but can't find one.

You simply run the "InstallService.bat" in the bin folder as Administrator account (while Aware is not running). Then you go to Windows services and start AwareIM. Be sure it is set to Automatic, so when Windows start, it will automatically starts Aware.

Note: Please check with support on memory settings for Aware and Tomcat when they are run as "service".

« Previous Page Next Page »