When you build a small corporate data center you are unlikely to be too concerned about the state of the Internet in you area. This is especially the case if you are building the data center in some population center (New York, Frisco, London, Paris, Singapore, Hong Kong, etc.)

There will be enough bandwidth and, if bandwidth actually is running out locally, some provider will doubtless step in  and boost it. It’s not the same if you’re building a very large data center that’s somewhere off the beaten track. There may not be adequate local bandwidth that meets your needs. You may have to do a deal with a provider. You will also need to make sure that there is enough built in redundancy in the service (in case a line gets severed) and that there are no potential bottlenecks between the main arteries of the Internet and your data center. You’ll need to care about the local Internet.

Networking costs are (according to the rough model I drew up) about 12.3% of total costs. Of course that includes all the networking equipment in the data center and as the model we’re using is based on a scale of about 50,000 servers there would be a vast array of networking kit, including monitoring software and devices. You will doubtless also include a high level of redundancy internally (or deploy big data center switches that have it built in.)

Economies of Scale

Let’s consider the economies of scale that you may be able to get here. First of all, with a very large data center you are going to be in a situation to negotiate with your communications provider. So you may buy a great deal of bandwidth, but you won’t be paying a high rate per gigabyte and you will be buying a guaranteed service.

Within the data center, you may be able to predict the network traffic with great accuracy. This depends on the kind of service the data center is providing. If the scaled data center is Infrastructure as a Service (IaaS) or Platform as a Service (PaaS), then it’s going to be less easy to do this.

Network traffic depends on what the applications do and if you don’t know ahead of time then it’s equally possible that you might have workloads that do nothing more than send short messages over the network (say 100 bytes) or you may have whole videos moving around (say 2 gigabytes). There’s a big difference there in bandwidth requirements. You will want to virtualize the network for sure (using Cisco’s NEXUS 7000 or Brocade’s DCX or similar high power switches.) That gives you the capability of reconfiguring the network on the fly, assigning the bandwidth available in any part of the network and changing it dynamically where necessary.

However, if you are doing Software as a Service, you will be able to predict very accurately what the traffic will be and the need fro flexibility is much lower. This means that the cost will be much lower. That is the beauty of the SaaS scaled out data center; the single workload. Absolutely everything can be designed for one application. When you have that situation, all the costs come crashing down.

See also:

The Death of the Data Center: The Model
The Death of the Data Center: Location, Location, Location
The Death of the Data Center: Power
The Death of the Data Center: Cooling
The Death of the Data Center: Networking
The Death of the Data Center: Server Hardware
The Death of the Data Center: The Software
The Death of the Data Center: Software Optimization