Server is dead in water

kklosson

My server will restart and last about 5 minutes then it just stops responding. Man I've looked at everything. I'll listed to any suggestions.

customaware

Hi Kinglsey....

Can you please provide some info...

What sort of server is it?
What is the config?
Where is it?
What's on it?

Is it backed up?

kklosson

It's on AWS. When it dies there is no load whatsoever.
It's backed up. My consider restoring a snapshot.

customaware

Are you using mySQL / RDS

customaware

One of mine did that Kingsley and I doubt it is you problem but worth checking...

I ran out of Hard Disk space..... which never occurred to me as I had not put anything extra on the Instance...

However, MS-SQL Log files grew and grew. Data was like 500 MB but the log file was over 70 GB. Once I shrunk it and
reconfigured then all good.

Anyway...check disk space

RLJB

Same experience a while ago. Check disk space, we had a back up file taking up space. Mysql wouldn't start.

kklosson

Great tips but I'm good on disk space - very good.
MySQL running on a stand alone AWS machine.

mrbdrm

could it be a memory leak? try to disable startup programs and services that run with the server.

kklosson

Thanks but no.

BenHayat

Just for clarification, when you say "My server", are you referring to Aware Server or MySQL server or Tomcat Server?
And what if you run Aware as Service, which I do all the time and runs with no issue (of course much smaller size).
Just a thought!

customaware

I have emailed you privately Kingsley

kklosson

The server started behaving normally on Monday though I had done nothing. I don't have any definitive answers for this but here are my theories...

Theory 1: The host machine in the AWS infrastructure had problems or was overstressed over the weekend. I don't know enough about virtual systems but I am constantly told that your neighbors can impact performance one way or another. Bear in mind I never got any error messages of any kind.

Theory 2: Since it started on Friday at midnight and seemed to end on Sunday, perhaps AWS engineers were doing something awful. Kind of goes with theory 1.

In order to do "something", I'm going to upgrade my AWS machine and reinstall AwareIM. This does 3 things... (1) moves me to a new neighborhood in AWS, (2) puts more juice on my machine, and (3) reinstalling software is always a good shotgun approach to reset the baseline. My next step would be to move to a dedicated server, which Support and others have recommended. But if I do, I should probably move the database as well, which is somewhat harder. I prefer to stay in AWS. If you have more advice about the value of a dedicated server, or the woes of a virtual server, please let me have it.

BenHayat

Take a look here.

https://www.hivelocity.net/dedicated-servers/#products

kklosson

Look in the domainversions table in basdb database in your backups. It holds BSV names and versions. Perhaps a clue.

kklosson

Sorry, ignore above, wrong topic.

kklosson

Thank you Ben, but I'm not so much shopping for servers. I'm looking for some objective data that supports moving to one.

customaware

Have you enabled Cloudwatch Kingsley?

kklosson

Yes. All metrics indicate the machine is fine, as I would suspect because the machine never indicated any issues. CPU, Mem, I/O, disk space all green during meltdown.

BenHayat

kklosson wrote
Thank you Ben, but I'm not so much shopping for servers. I'm looking for some objective data that supports moving to one.

The person who actually sent me that link, was (still is using S3) AWS EC2 user for his SaaS application. He started with AWS way early with their S3 storage for digital documents (that's what he stores and retrieves). Under load (users uploading or downloading large files) the AWS EC2 would get very slow to respond, especially during 9-5 hours. He moved to bigger VPS, separated database and etc and still had issues.

He finally moved to dedicated server, with SSD disks and very large bandwidth, and all his problems were history and never looked back. He said AWS was great for quick bursts to execute something for short period, but if you need long running jobs or long process or intense CPU for long time, the VPS can't keep up and you'll see lots of ups and downs.
Now he uses S3 for storage only and his SaaS app runs on a dual processor with another one as backup, if this goes down. Yes, he pays more money, but he charges more for better service he provides. He told his customers, if they want faster responds, the price will be %x more and they agreed.

kklosson

Thanks Ben. That's good info.