Scaling up/out in Aware? Best practises

[hpl123]

Hi all,
I am potentially faced with a scaling situation where I need to be able to scale AwareIM applications "up" (and preferably "down") in an effective way and starting this thread to try to get a discussion going on scaling AwareIM applications and also to try to flush out some best practices on this topic, recommendations from people doing this or from people who have an experience with scaling whatever infrastructure.

The 2 main approaches for scaling are scaling up/vertically which means adding more resources to an existing server or scaling out/horizontally which means adding more servers in parallell. From what I have read, scaling out is recommended for most scenarios IF:
1) The situation requires it i.e it´s not a single webapp growing over time but rather a situation where high traffic demand, spikes, rapid growth etc. can occur.
2) The application can handle it i.e it´s possible to scale out, application source and all things can be cloned to another server without configuring too much.
3) Using a cloud based infrastructure where servers can be added/removed in a simple, dynamic way.

Questions I have about scaling out in Aware:
- Is anyone doing this? Have done it?
- When it comes to Aware licenses, this can be a problem. How should one solve this? Are there any workarounds? In worst case scenario, I guess it should be possible to buy a couple of extra licenses and have them IF/WHEN a new server has to be added and then it can be quickly configured? Other things to think about?
- Thoughts on best cloud based infrastructure that supports scaling? AWS? UpCloud? Others?
- Separate database server (s)? I suppose a dedicated DB server is ideal, should this be set up to scale up possibly i.e adding more resources to the one DB server as needed? Will that be enough or is it better to also set up a scale out solution for this as well? How many application servers can a single dedicated DB server handle?
- Load balancing? Do the cloudbased infrastructure solutions include a load balancer? Recommendations or thoughts?
- Scale down? The infrastructure should be able to scale down easily, quickly etc.. Is this possible in cloudbased infrastructure solutions? Is the price based only on use e.g usetime so it´s not a 30 day contract or something like that? Things to think about?
- Is it a manageable process to do this either automatically or manually i.e how difficult is it to work with it once set up?
- Thoughts on costs, where to save and optimize etc.?
- Is there a "upper limit" to how much an AwareIM scaled solution can scale? If for example scaling out, is it possible to add servers ad infinitum and the Aware app can scale ad infinitum?

ALL thoughts, tips etc. are appreciated and I hope this post can act as a guiding document for how to handle scaling in Aware. PS: I would also gladly jump on a call or Skype call or whatever if someone would be willing to do that.

[hpl123]

Does anyone know how Tomcat handle concurrent users, threads etc.?

I have read some articles here and there about it and think this is how I understand it to work but not sure:
- Tomcat has a max threads e.g 200 threads (can be configured)
- Threads are located in a thread pool where each thread has a free VS active status
- If a webapp/user is doing something e.g clicks to make the application do something, a free thread is assigned and the status is changed to active thread after which the application does whatever the user wants. When the task is done, the active thread is released and the status is changed back to free thread. If all 200 threads are active, new requests are placed in a queue and Tomcat picks the requests in the queue as soon as one of the tasks from the active threads is completed/released to free thread. Is this how it works?
- I have also read about "keep alive" threads where threads are kept in a active state, I hope that isn´t how it works in Aware (and if it is, why?). Anyone knows?
- What is a connection in this context? Is it a request being made or something else?
- What is the difference between session, connection and thread?

Except for the questions above, the age old question. How many concurrent users can a Aware app handle, any new insights from anyone? Yes I know it depends on app, server etc. etc. BUT still it should be possible to estimate somewhat for a medium complex application doing no long process operations or such.

[Rem]

Can you give us some more specific information regarding your situation?

Like number of users right now, what kind of system our are building (public self-register, inhouse company ...) and how many users you are expecting and how fast they have to be onboarded? More companies or more clients in the same company etc.

Also, how about upgrades to the system, how often do you plan to push upgrades.

We have similar questions, but right now we are developing and are HOPING the our application will take of (so we will not have the "problem" otherwise)

[hpl123]

Can you give us some more specific information regarding your situation?

Like number of users right now, what kind of system our are building (public self-register, inhouse company ...) and how many users you are expecting and how fast they have to be onboarded? More companies or more clients in the same company etc.

Also, how about upgrades to the system, how often do you plan to push upgrades.

We have similar questions, but right now we are developing and are HOPING the our application will take of (so we will not have the "problem" otherwise)

I can share some information but is a complex scenario so just sharing some overarching details.

Background:
The application is for the tourism sector and has a public facing part that in theory can explode with users and/but only during peak times here and there and primarily during the summer season so the main problem I need to plan for is peaks in users and how scale up/down the infrastructure. I have run a successful pilot project with the product on a small scale and now "taking it to market" or whatever you want to call it and as part of that we are currently in negotiations with 4 municipalities where the app can be rolled out as part of a "kickstart tourism" campaign (after Corona) and depending on how big they want to make it, it can become gigantic with many thousands of users (peaks). We have not closed the deal yet and as it is a municipality/state affair, they have a lot of hops we have to jump through, estimate costs and contingencies etc. etc. and that is basically what we are trying to do now and to do that, we need to plan/estimate a infrastructure and scaling plan and Aware is the core component in that planning.

Infrastructure:
The infrastructure we need to set up is basically a base infrastructure which consists of 1) Aware application server, 2) database server and 3) IIS static content server. This is the base infrastructure (1 per city so can be many different base infrastructures) and then the base infrastructure needs to be able to scale up adding more Aware and IIS servers and scale down by removing Aware and IIS servers. The scaling up/down is based on demand of course and the main problem again is peaks in users. I only have experience working with dedicated servers and slowly growing the app in size etc. and/but for a situation like this, we need to use a cloud hosting platform where we can add servers automatically or in a simple way to cover peaks and then of course after the peak, we need to scale down again so we don´t pay for the entire infrastructure during the entire month or year.

Problems:
One of the main problems is uncertainty, we have no clue how many users we might get during peaks and all depends on scope e.g how many cities in the municipalities are part of solution? all municipalities? distribution/marketing, how will this be distributed etc. etc.?. Another uncertain element is, how many concurrent users can a single Aware app handle? We also need to have a plan for the scaling of Aware (license problems etc.), scaling IIS is easy. Part of this is also of course setting up the entire infrastructure (s) with a cloud hosting platform and all it entails (load balancing, which cloud platform is best etc. etc. etc.).

I am in dialogue with Vlad about the entire situation so feel at easy knowing he is in my ringside but still, it´s a lot of moving parts needing to be thought through and planned/estimated and then we´ll see where it leads. If we don´t close, we are still taking the product to market but then we won´t have problems like this to think about, at least not initially.

Any input, tips etc. are appreciated.

PS: Rune, I would also be interested to hear more about your app and situation?

[swiftinitpvtltd]

We did build one aware based web app last year that had at one point 1200 users and 250+ concurrent users. It worked perfectly fine on strong single production server with 200 users doing some kind of activity on same day. I am also interested in seeing scaling tips for large scale app with 50k or 100k+ users in frontend based web app like amazon, Eventbrite.

[hpl123]

We did build one aware based web app last year that had at one point 1200 users and 250+ concurrent users. It worked perfectly fine on strong single production server with 200 users doing some kind of activity on same day. I am also interested in seeing scaling tips for large scale app with 50k or 100k+ users in frontend based web app like amazon, Eventbrite.

How is this app doing now? What was the server specs of the server used?

[hpl123]

Ideal server specs for base server? Any thoughts on what the ideal specifications i.e RAM, CPUs etc. etc. should be for a starting (core server which then when resources are not enough would be cloned) Aware server. The goal is of course to handle as many concurrent users as possible. Please chip in people with thoughts or experience?

[hpl123]

Related discussion: https://www.awareim.com/forum/viewtopic.php?f=1&t=11885

[hpl123]

Does anyone know how Tomcat handle concurrent users, threads etc.?

I have read some articles here and there about it and think this is how I understand it to work but not sure:
- Tomcat has a max threads e.g 200 threads (can be configured)
- Threads are located in a thread pool where each thread has a free VS active status
- If a webapp/user is doing something e.g clicks to make the application do something, a free thread is assigned and the status is changed to active thread after which the application does whatever the user wants. When the task is done, the active thread is released and the status is changed back to free thread. If all 200 threads are active, new requests are placed in a queue and Tomcat picks the requests in the queue as soon as one of the tasks from the active threads is completed/released to free thread. Is this how it works?
- I have also read about "keep alive" threads where threads are kept in a active state, I hope that isn´t how it works in Aware (and if it is, why?). Anyone knows?
- What is a connection in this context? Is it a request being made or something else?
- What is the difference between session, connection and thread?

Except for the questions above, the age old question. How many concurrent users can a Aware app handle, any new insights from anyone? Yes I know it depends on app, server etc. etc. BUT still it should be possible to estimate somewhat for a medium complex application doing no long process operations or such.

Talking to Vlad and isn´t as easy as this (the Tomcat thread/queue things) as there are other things involved e.g DB transactions and things i.e it is complex. It would be great if there were some way, tools etc. to actually test Aware app and infrastructure performance. Then we could look at OK for this app, the following scenario creates x load and can handle y numbers of concurrent users etc. etc..

[johntalbott]

Talking to Vlad and isn´t as easy as this (the Tomcat thread/queue things) as there are other things involved e.g DB transactions and things i.e it is complex. It would be great if there were some way, tools etc. to actually test Aware app and infrastructure performance. Then we could look at OK for this app, the following scenario creates x load and can handle y numbers of concurrent users etc. etc..

There are many. Here are a few examples.

Load Testing tools
https://theqalead.com/tools/load-testing-tools/

Application Performance Management (APM) tools are intended to monitor every layer to identify bottlenecks.

https://www.appdynamics.com/supported-t ... ee%20trial

https://newrelic.com/platform/full-stack-observability

https://azure.microsoft.com/en-us/services/monitor/

[hpl123]

Talking to Vlad and isn´t as easy as this (the Tomcat thread/queue things) as there are other things involved e.g DB transactions and things i.e it is complex. It would be great if there were some way, tools etc. to actually test Aware app and infrastructure performance. Then we could look at OK for this app, the following scenario creates x load and can handle y numbers of concurrent users etc. etc..

There are many. Here are a few examples.

Load Testing tools
https://theqalead.com/tools/load-testing-tools/

Application Performance Management (APM) tools are intended to monitor every layer to identify bottlenecks.

https://www.appdynamics.com/supported-t ... ee%20trial

https://newrelic.com/platform/full-stack-observability

https://azure.microsoft.com/en-us/services/monitor/

Thanks John and yeah I am looking into a couple of these. I have no clue how to set them up for Aware though and never used these types of tools before so kind of stumbling in the dark. Have you tried a tool like this and/or is using one?