A sneak peek into the Service Provider landscape

Udayakumar Nalinasekaren
Dec 19, 2010

We talked about Gmail. We know that millions of people around a country use Gmail service. They all access the application that runs on a computer inside the cloud. During a typical day, it is easy to imagine lean periods of Gmail usage where only a handful of people are consuming Gmail services. It is equally easy to imagine a time of the day where hundreds of thousands of people could be simultaneously checking e-mail.

The obvious thought for the cloud service provider is to build one huge computer to handle peak load to ensure that all customers get responsive service. Imagine the size and scale of that computing machine. It has to be several times the power of the personal computer you see at home. Imagine the amount of power it will consume. Also imagine the idling CPU, memory, networking and disk capacity when just a handful of users are on the service. An idling computer still consumes a huge amount of power. Obviously, this approach is not useful to the service provider.

What would be an alternate strategy? It would be nice if the cloud service provider has several home sized machines that are capable of handling a few hundred users at a time. Each is a perfect replica of the other. As the number of users increase, the service provider powers up additional machines. When the consumption goes down, the service provider powers down those with no load. Sounds better already but think carefully. The PC box we are familiar with has a disk spinning in it and therefore has program and data stored inside the box. If we use such a box, then specific users will be assigned to specific boxes where their email data is stored. Depending on which user wants to check e-mail. his specific box needs to be up and running. If we have only one hundred users checking mail and it so happens that each of their respective mail is in a different box, we will need to run one hundred boxes. Each of them will be running at a hundredth of its capacity. We are back to square one.

Let us use one more trick. Seperate the disks from the boxes and make an arrangement whereby any box can access any disk. This is technically called a Storage Area Network or SAN. Sounds like we have a solution but not yet.

If we do not hardwire a user to a specific computing box, then there must be a way to direct the user when he or she comes in to check e-mail to the next available computing box. If all the current boxes are running to full capacity, we need to startup a new box quickly while we keep the user on hold and direct the user to that box. This is called Load Balancing. Load balancing also helps consolidate available users to a minimal configuration of boxes as the number of users start diminishing. The unused boxes can be switched off.

The figure below is a very crude illustration of a Service Provider setup inside a cloud.

So a cloud service provider should minimally have diskless boxes, a storage area network, load balancer and networking technology to connect everything together. Only such a setup will enable the service provider to effectively leverage economies of scale and scope.