Memory Issue - huge RAM consumed even no users logged in

[weblike]

After 2 days of using it I can say is not entire ok.
Afrer cleaning the memory I had cases when the app just not responded for 45-60 seconds and I had to restart Awareim.
Maybe it's only my system. Anyone can test this?
Thank you

[BenHayat]

After 2 days of using it I can say is not entire ok.
Afrer cleaning the memory I had cases when the app just not responded for 45-60 seconds and I had to restart Awareim.
Maybe it's only my system. Anyone can test this?
Thank you

I think you're trying to solve a problem in a wrong way. As if someone has an ulcer and keep taking aspirins...

That tool was good to "tell you", your app need optimization and has efficiency problem. That's where you need to focus on and not a bandit solution.

[ACDC]

George
Is it possible that you have a single reference on an object set to "drop down" that has "Fetch all records at once" set ?
(I once had a similar problem that manifested over time where a product object grew to thousands of records and whenever multiple users clicked on the drop down, thousands of records were loaded into memory multiple times using up all memory space resulting in hanging)

[weblike]

George
Is it possible that you have a single reference on an object set to "drop down" that has "Fetch all records at once" set ?
(I once had a similar problem that manifested over time where a product object grew to thousands of records and whenever multiple users clicked on the drop down, thousands of records were loaded into memory multiple times using up all memory space resulting in hanging)

Thx for the tip...I will try to use the other option...

[tford]

George
Is it possible that you have a single reference on an object set to "drop down" that has "Fetch all records at once" set ?
(I once had a similar problem that manifested over time where a product object grew to thousands of records and whenever multiple users clicked on the drop down, thousands of records were loaded into memory multiple times using up all memory space resulting in hanging)

It sure would be interesting & helpful to develop a list of potential config tool "warning" conditions that could be displayed for a BSV.

Let's say you have two attributes that have a "drop down" setting as ACDC described above .. one that makes total sense and the other that has grown out of control. In a perfect world, I'd love to be a list of these potential design concerns on a screen with the ability to flag the conditions you don't want to continue to display time after time. Of course, you would also need the ability to see the entire list (including items you have previously flagged to no longer display) so that you would have the ability to do perhaps an annual deep dive into a BSV for potential design concerns.

The Cadillac version of a tool like this would allow a menu option in an operational BSV (for BSV designers to use) that actually looked at conditions in the database (in this case # of records in a particular "fetch all records at once" drop down) and show a list of concerns for potential action.

[4xjbh]

George,

I am curious, did you resolve this, what was the solution?

[weblike]

Hi,

Didn't have time to optimize my application yet (like Vlad's suggests in one of his video).
For the moment I restart the services every morning (AwareIM server, mysql server) and I empty the table "execution_contexts". That seems to be enough for the moment.

What concerns me is that, in the end of the day I have zero users logged inside the app and the RAM is not going down (keeps staying in huge RAM).

I will keep this monitoring...and soon I will start to optimize my app, because I have mostly 20-30 concurrent users and the RAM is staying at 6-7GB.

Cheers,

[weblike]

I have followed the optimization video and managed to eliminate recursive rule execution, but I cannot bring the RAM under 6GB.
My system is only loaded with maximum 20 concurrent users at a time. I'm scared of what happens if 1000 enter in the system (in my dreams )?

Selection_013.png (15.85 KiB) Viewed 20431 times

Selection_014.png (33.41 KiB) Viewed 20431 times

I guess some processes are remaining stuck because even NO user is logged in, the RAM remains above 6GB.

I'm out of ideas here...

[4xjbh]

You may just need to get a ticket from support and have them review the system before your users lose confidience in the application. Sooner the better.

[weblike]

You may just need to get a ticket from support and have them review the system before your users lose confidience in the application. Sooner the better.

Maybe you are right.
The thing I don't understand is why the RAM doesn't go down when NO users are loggedin. It's Saturday morning, No user is working in the app and here are the stats:

Selection_015.png (19.24 KiB) Viewed 20423 times

Selection_017.png (14.76 KiB) Viewed 20423 times

[weblike]

Hi all,

Managed to go from 6.5 GB to 3.5 GB

What I did is:

1. Optimized the rules only from 1 BO from my app (following the Vlad's video). This BO is the mainly BO used in my app.
2. In Aware server output I have seen that it's trying to send the UNSENT_EMAIL messages. For very long time I didn't cleaned up that folder, so I did now.
3. Truncate the execution_contexts table
4. I'm using the clean_mem tool every morning with scheduled task.

I still see in the Aware Server output these messages which I don't know what they mean. Anyone can help?

INFO: OutputAdaptor1.Controller has commited 92900 transactions
INFO: InputAdaptor1.Controller has commited 92400 transactions
INFO: OutputAdaptor1.Controller has commited 93000 transactions
INFO: InputAdaptor1.Controller has commited 92500 transactions
INFO: OutputAdaptor1.Controller has commited 93100 transactions
INFO: InputAdaptor1.Controller has commited 92600 transactions
INFO: OutputAdaptor1.Controller has commited 93200 transactions
INFO: InputAdaptor1.Controller has commited 92700 transactions

thank you

[RLJB]

George - I feel your pain brother.

Currently nursing a prod server for the past 4 hours, trying to figure out why it is today going as SLOW as possible. Whereas in the past it has been fast. The suggestions above are ones we regularly follow:

1. Clear UNSENT_MAIL folder
2. Clear Execution Context table
3. Optimise App as per Vlad video
4. As ACDC pointed out, drop downs can be a real killer, watch them and make sure you are not 'fetching all' if there are a large number of instances in the BO, this is very hard to find as it will not show in any logs
5. Watch out for attaching large docs to the RegularUser BO, this can slow users down

All of this raises one important issue we have discovered, the logging currently makes it VERY hard to find these kinds of problems. We have a simple Linux memory too called "Top" to measure memory usage, we can track this per second to see memory spikes. BUT you can't easily track what a user is doing in your app with current logging.

It would be AWESOME if there was some form of Aware logging that simple output to a text file:

ServerTimeStamp | BSVName | LoginName | Useraction

where the 'Useraction' in the above is what the user is doing in your app, it would be:

- the form name, process name, button name they clicked
- the process name that was initiated from a user action

basically "what" the user initiated at that time.

If you had this info, you can map memory spikes to the user interaction by timestamp and zero in on the problem immediately.

The current logging is either far too detailed and full of Java items that you don't understand and detailed rule terms, OR not detailed enough. Having said that, I still want to keep the current logging, I just want an additional piece as described above