SaaS elastic scaling?

kklosson

I think a tutorial on this would be nice and would be demonstrative of AwareIM's ability to scale. Here's my primary consideration regarding scaling based on my experiences. I've seen many times where one or other Java processes break down irrespective of the machine load. There are many ways to setup load balancing but at its basic level, the load balancer is monitoring the machine's CPU, RAM, or other factors. No matter how much RAM your machine has, only a certain amount of it is allocated to the two Java processes. When either of them run out of memory, the load balancer is not going to know unless you have it configured to monitor Java processes at some level, and there are tools for doing so, but they are complicated (for me) and I haven't messed with them. I've suffered a number of system failures which have been noted on this forum. In all cases, my machine was never stressed - it was the Java processes that failed. At this point, my app server is running superbly and I literally have no idea where the breaking point is, or what it is, but I don't believe it will be the capacity of the machine. Theoretically, in 64-bit operation, you should be able to feed the Java processes maximum RAM and if your CPU can keep up, you will be serving a lot of customers before something starts to break routinely. The question then will be what is breaking? Under a very heavy load, your system will simply take longer to process requests or one of the Java processes will be failing (probably due to memory starvation) and it will be time to get a second server and a load balancer. If you can't afford to run the load balancer around the clock, you will have to devise ways to determine when to turn it on and when to shut it down. If you must provide service at the 5 9s level, there is a solid case for at least two or more application servers purely for maintenance and system availability.

My last tip is this. AwareIM is fantastic, but ultimately, you have a responsibility to architect your app in such a way that it can operate at scale. If you think you're going to need a server farm, you have a lot of up-front considerations and you must think about operations at scale in everything you do. Be sure to watch the tutorial on optimization and consider your alternatives in designing objects and relationships. A good example is Ben Hayat's recent post https://www.awareim.com/forum/d/9042> in which he is looking for the optimal design of the RegularUser object. This is the kind of thinking you must apply (ask me how I know...).

To exact point of your question, you should be able to scale your app server up/down fairly rapidly by maintaining a static snapshot of the server and provisioning it to a larger or smaller instance - no license issue there. This should only take a few minutes, but there is downtime for sure. I don't do this with any regularity, but at times when I have opted to scale up, that's what I did and it's pretty easy. If well-planned, you should be able to do it in a few minutes.

MichaelQ

Thanks for your really useful replies.

I'm a newbie at this, but it has become a very important subject. Talking with my client it now seems that they are placing greater emphasis on performance and the potential to handle higher traffic loads without breaking down. This is very worrying:

I've seen many times where one or other Java processes break down irrespective of the machine load.

It would indeed be very helpful if Awaresoft could provide a video to explain more about cloud performance and scaling options.

It would also be very reassuring if Awaresoft could provide some performance metrics indicating how a standard optimized app could be expected to perform on standard infrastructure. For example perhaps they could use one of their sample apps and install it on a standard AWS instance and then do some standard stress tests. They could then use the metrics gleaned to quantify various improvements such as:

Further optimization of the app.
Upgrading to a higher performance AWS instance.
Adding a load balanced server & alternative load balancing methods.
Is there a way to be alerted before an app dies so that preemptive action can be taken?

This could also be used to measure how AIM performance (hopefully) improves between releases.

As things stand, I have absolutely no idea what costs I'm going to incurr for servers & AIM licenses. particularly if it seems that AIM might not use server resources efficiently.

The laudable and very attractive goal of Awaresoft is to abstract away most of the arcane technical hassles, which it seems to mostly do very successfully. Perhaps the trial version can be used for stress testing , but I'd suggest that prospective developers shouldn't have to buy a license and then do all this load testing themselves just to find out whether AIM might work for them or not. Some actual metrics from Awaresoft would be a great help.

Michael

kklosson

I hear you, brother, but one developer's AwareIM app will scale and perform very differently than another's. To me, it's less about AwareIM and more about Java itself. To operate a high-volume AwareIM application, one must develop some expertise in managing Java and becoming acquainted with some of the Java-related performance management tools like New Relec and others than enable you to view Java internals, in particular, memory management. You started by saying that you would build a simple app. To me, simple means a relatively small number of objects with a relatively small number of attributes, and no complex queries, business rules, or processes. Developing a "simple" app will give you less to concern yourself with in scaling and optimizing, but I would consider my app to be "complex" and I have no concerns about scaling or performance at the moment.

BenHayat

I think for starter, let's have a solid document and videos how to setup an scalable configuration and what softwares we need to setup for load balancing and how a load balancer talks to Aware server to know Aware's state of load.

We really need proper guidelines and best practices how to this. Seems like different users have done different ways, but we don't know who is doing what and if their approach is correct.
Just saying "Aware IM is quite suited for elastic scaling", doesn't really help us, with no guidelines and documentations and is more of a marketing statement. How is this statement backed up with actual steps of implementation?

hpl123

1+ for a proper technical implementation document for scaling and load balancing.

aware_support

This is very worrying:
I've seen many times where one or other Java processes break down irrespective of the machine load.
....
To me, it's less about AwareIM and more about Java itself.

Please don't generalize. If you have some issues in your application it could be due to thousands of reasons - from your server resources to the details of your application.

THERE IS ABSOLUTELY NOTHING WRONG WITH JAVA (or Aware IM for that matter). Java is used in millions of applications and billions of devices.

This issue has been discussed many times. Asking how Aware IM will perform is akin to asking how Java or C++ will perform. The answer is - IT DEPENDS ON HOW YOU CREATE YOUR APPLICATION. With the greatest of tools you can create something that will bring any server to its knees.
Please watch the Optimization tutorial available on our web site for more details.

As far as Aware IM is concerned - it is fully multi-threaded, transactional, robust and scalable. Period.

There is absolutely NOTHING special you need to do (apart from licensing) to support scaling as Aware IM has been designed with scaling in mind. You just need to install another application server and fix licensing. The cost of extra server is $1099

Having said that, we will add a special tutorial about scaling. However, please watch the Optimization tutorial first - it will explain a lot of things about scaling as well.

BenHayat

aware_support wrote

Having said that, we will add a special tutorial about scaling. However, please watch the Optimization tutorial first - [b]it will explain a lot of things about scaling as well.[/b][/quote]

Thank you Support for the considerations.

Way back when the optimization video came out, at the very end when you had suggested that to have multiple Aware servers, we need to "Statically" split users to different servers via different URLs. This option might be a good option for "[b][u]Intranet[/u][/b]" where there are static # of known users to be assigned to different servers, but in the world of Internet and SaaS applications where users come and go, is almost impossible to maintain.

When you're putting a tutorial together, please be sure to give us a comprehensive and "[b]Dynamic[/b]" solution where all users are sent to ONE URL and then depending on the load on each Aware server, the "[b]Solution[/b]" figures out which Aware server to use.

The second issue that comes to mind, is how the front end Load Balancer is going to maintain the same user that has already started with one server, will continue on the same server during their session and subsequent requests?

Thirdly, we also need to know how multiple Aware Servers will be connected to a single DB Server, i.e. AWD RDS server or another server running.

I think these details, are what most people are struggling with and it would be great to make sure a sound and uniform solution is offered by Awaresoft, not only for existing customers, but for future prospects to see and know, AwareIM CAN be used in a scalable fashion. The time invested on this document/tutorial will be very valuable and requires a through coverage.

[b]Once again, thank you![/b]

MichaelQ

It would also be very helpful if:

there was a way to enable AwareIM to "fail gracefully." In other words when a server becomes overloaded it shows a static page explaining that the app is temporarily unavailable.

there was a way to alert the developer when say 80% load is reached, so that remedial steps can be taken.

I had a look at New Relic and this seems to be a very useful tool for scaling and performance.

Michael

kklosson

To be fair, I don't believe you would find such a feature in other dev environments or frameworks. Tools such as New Relic may become a necessity for you at some point if you suffer failures, which will most likely point to something in your app's behavior - as opposed to AwareIM or Tomcat simply "breaking", which they don't do without some reason. I went through a long period during which my app was crashing rather frequently. The symptom was that the server machine was fine but Tomcat (or AwareIM) simply died. It was very difficult to diagnose and understand and a tool like New Relic would have helped if I knew how to install and use it. The lesson out of all of that was to consider performance at every step of design and implementation. I resolved my issues by optimizing my application, which included optimizing business rules and using some client side scripting for certain functions that were taxing the server. In all likelihood, your first application crash will be caused by memory starvation in a Java process (Tomcat or AwareIM), caused by something you did or didn't do. Your machine's ram and CPU will be all green so you will have use a monitoring tool to fully understand what is causing it.

MichaelQ

I know that each app is different and will scale differently, but are you able to share some metrics about your app? (Specification of server, max concurrent users etc).

Also suppose you have a server which runs several entirely separate apps. How does one manage scaling and load balancing in this case?

Michael

BenHayat

MichaelQ wrote
Also suppose you have a server which runs several entirely separate apps. How does one manage scaling and load balancing in this case?

Michael

If one is at a point of need for Load Balancer, that means the app requires multiple Aware server. You don't want to have your aware server to run multiple apps. You want an app to be run by multiple aware servers and not the other way around.