Dear All,
I found this article on Oracle/MySql site relating to scaling PHP/MySql applications (http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html). I’m wondering if any of you have used these techniques or others with ScriptCase to scale your solutions.
Any thoughts welcome.
Thank you!!
Lyle
Hi Lyle,
First, I applaud you looking at this issue. I have posted in the forum asking similar questions. Here is my experience so far, from my perspective.
I use a hosting service that is very programmer friendly - Webfaction - and they have some load balancing and other features that are worth looking into for scaling and handling huge spikes in load. I have used them for many many years, with both Pytrhon and PHP based systems, and I currently run a couple of pretty big systems on them. Like with literally 100+ mysql DBs and such. I have had like only 2 short down periods over the past 6 years. They are also super responsive 24/7 at the support ticket center. Those guys at Webfaction have PHP experts on staff, and they get answers back very quickly. If I were you, I would send them an info request. You can get an eval account I believe too. If you use this link and sign up , I will get credit for the referal FYI, but I use them myself so I stand by them 
http://www.webfaction.com?affiliate=insmalls PS I you don’t want me to get credit - just go to webfaction dot com…
Anyway, I have run both load balancing and a system using multiple mysql DBs. For load balancing, you could use the system Webfaction has or just install a free instance of pound - pound is an open source load balancer, so you could have it basically direct part of the traffic across multiple instances of your system. So for instance, you might make an SC project called Project X with 10 apps, but install the deployment of that project X into, say, 3 webfaction application/websites… then install pound into its own application/website on webfaction, and point your domain names there. Pound then switches the traffic and supplies the webpages from your apps automatically, so that is for some reason one was down the others can carry the load. Pound has a TON of features that lets you do all sorts of things.
Now that solution can help with load, but them are all 3 of your Project X deployments feeding off 1 DB? That can be fine, and is the easiest. I have never had a mysql db choke or crash on Webfaction - period. I make backups daily, but have never had to do a crisis restore (just tests because I am paranoid and been burned in past jobs). They also do backups, but just to recover a server in case it blows up - their fault. But is you have sensitive data (who doesn’t?) , run your own.
I dump mysql DB and store the .sql files. I am looking at some third party remote operating systems, that could run an server anywhere, that can run dumps and download to a storage system somewhere else - or do a replication process to put into a hit standby DB.
I am kinda leery about the replication - in that there may be a lag is someone did a transaction on one system and it didn’t show up right away on another… but I did check a month ago with webfaction and you can set up replication there as well. My personal solution is to run one db, BUT have that one dumped and copied every hour into a hot standby I could switch to… on a second webfaction server. It is also possible to pay webfaction to put you onto a server with very limited number of other users, although I have not had to do that yet. They also can put you on a totally dedicated one too. I feel like I can scale where I am.
The other thing is to look at making a static version of your site, if that is possible, so there is no PHP running at all - or very limited where needed. You can google web crawlers and use free ones to make static version - generated originally by your apps. Static HTML content always beats any generated code pages. I have done that before as well. But honestly, I don’t do that right now.
You can also look at using Amazon to host DBs or systems, on servers which are really virtual machines - but you can pay more and more and rev up the specs on the fly. In theory, I believe you could host your own massive site that way that would be like Facebook or Zappos shoes. But there is complexity and $$$$$$ involved. I have only played there. You can fire up stuff and play for pretty cheap.
So anyway, I am more worried about failures and making sure I have hot standby server and DB, so I can re-point my domain names, etc. I ave found the most problems I have experienced were due to things like PayPal changing something in their API that ran for years - and not knowing - then having a zillion help requests come at me. Those 3rd party issues suck. If you host with someone like webfaction, server down-times should be minimal, and you can communicate with help desk people. Something bigger, like PayPal - forget it 
I am not sure what Amazon support is like…
That is my take on things - keep posting as you find out more.
Peace,
Jamie
Good Points
Hi Jamie
Thank you for your in depth response. I’m thinking though it. WebFaction does have good content about their load balancing and definitely looks developer friendly. I’m going to study more about load balancing and get back to you in a few days.
Thanks so much!!
Lyle
[QUOTE=onmountain;16845]Hi Lyle,
First, I applaud you looking at this issue. I have posted in the forum asking similar questions. Here is my experience so far, from my perspective.
I use a hosting service that is very programmer friendly - Webfaction - and they have some load balancing and other features that are worth looking into for scaling and handling huge spikes in load. I have used them for many many years, with both Pytrhon and PHP based systems, and I currently run a couple of pretty big systems on them. Like with literally 100+ mysql DBs and such. I have had like only 2 short down periods over the past 6 years. They are also super responsive 24/7 at the support ticket center. Those guys at Webfaction have PHP experts on staff, and they get answers back very quickly. If I were you, I would send them an info request. You can get an eval account I believe too. If you use this link and sign up , I will get credit for the referal FYI, but I use them myself so I stand by them http://www.webfaction.com?affiliate=insmalls PS I you don’t want me to get credit - just go to webfaction dot com…
Anyway, I have run both load balancing and a system using multiple mysql DBs. For load balancing, you could use the system Webfaction has or just install a free instance of pound - pound is an open source load balancer, so you could have it basically direct part of the traffic across multiple instances of your system. So for instance, you might make an SC project called Project X with 10 apps, but install the deployment of that project X into, say, 3 webfaction application/websites… then install pound into its own application/website on webfaction, and point your domain names there. Pound then switches the traffic and supplies the webpages from your apps automatically, so that is for some reason one was down the others can carry the load. Pound has a TON of features that lets you do all sorts of things.
Now that solution can help with load, but them are all 3 of your Project X deployments feeding off 1 DB? That can be fine, and is the easiest. I have never had a mysql db choke or crash on Webfaction - period. I make backups daily, but have never had to do a crisis restore (just tests because I am paranoid and been burned in past jobs). They also do backups, but just to recover a server in case it blows up - their fault. But is you have sensitive data (who doesn’t?) , run your own.
I dump mysql DB and store the .sql files. I am looking at some third party remote operating systems, that could run an server anywhere, that can run dumps and download to a storage system somewhere else - or do a replication process to put into a hit standby DB.
I am kinda leery about the replication - in that there may be a lag is someone did a transaction on one system and it didn’t show up right away on another… but I did check a month ago with webfaction and you can set up replication there as well. My personal solution is to run one db, BUT have that one dumped and copied every hour into a hot standby I could switch to… on a second webfaction server. It is also possible to pay webfaction to put you onto a server with very limited number of other users, although I have not had to do that yet. They also can put you on a totally dedicated one too. I feel like I can scale where I am.
The other thing is to look at making a static version of your site, if that is possible, so there is no PHP running at all - or very limited where needed. You can google web crawlers and use free ones to make static version - generated originally by your apps. Static HTML content always beats any generated code pages. I have done that before as well. But honestly, I don’t do that right now.
You can also look at using Amazon to host DBs or systems, on servers which are really virtual machines - but you can pay more and more and rev up the specs on the fly. In theory, I believe you could host your own massive site that way that would be like Facebook or Zappos shoes. But there is complexity and $$$$$$ involved. I have only played there. You can fire up stuff and play for pretty cheap.
So anyway, I am more worried about failures and making sure I have hot standby server and DB, so I can re-point my domain names, etc. I ave found the most problems I have experienced were due to things like PayPal changing something in their API that ran for years - and not knowing - then having a zillion help requests come at me. Those 3rd party issues suck. If you host with someone like webfaction, server down-times should be minimal, and you can communicate with help desk people. Something bigger, like PayPal - forget it I am not sure what Amazon support is like…
That is my take on things - keep posting as you find out more.
Peace,
Jamie[/QUOTE]
Great Lyle, glad it was helpful. This is a topic that 95% of programmers tend to ignore. Maybe it is because most of us never realize the type of visitor/user loads that we dream about? In any event, Webfaction does have lots of great docs and their forums are full of related stuff. I found this post that talks about a load impact analysis site. Sounds pretty cool. http://community.webfaction.com/questions/393/caching-in-wordpress-best-practices
I look forward to what you find. I am always hoping and preparing for a great onslaught of users - then I can retire 
Peace,
Jamie