Choosing a data centre

When I set out to choose a data centre I knew I had a lot to learn to make an informed choice. Plenty of research later, and an amount of time studying Electrical Engineering at University, helped me reach a decision. This article is a reflection on those decisions and how I came to them.

Data centres themselves are complex monoliths representing huge capital investments for businesses who need to sell rack space to recoup that investment.  Starting to choose one brings you many decisions to make based on the business you are trying to support. Your Board might have given you a budget, or perhaps you have to set one, either way they are going to think there servers are now in an indestructible fortress so you had better provide one for the right price.

So what does a data centre provide? Well typically a data centre will provide some or all of the following:

• Power
• Cooling
• Physical security
• Fire suppression
• IP Transit (read: networking)
• Physical redundancy of systems
• Remote or “Intelligent” hands
• Managed Hosting
• 24/7 access
• Location Location Location

Some of these maybe be crucial to your business but all of them need consideration if your going to work out which facility is appropriate your company.

Power

Power consumption ultimately will determine the price you will pay for your hosting it also might rule out some data centres which can’t provide you with what you need. As a comparison between data centres this is the thing you should be looking at.

The first thing you need to do is work out what exactly you are going to put into the data centre and what power requirements it will have. Be careful here that you don’t look at the peak power of your server/switch/unit as that will not tell you what it will draw usually but what it will draw under extreme load.  Adding this up and coming to a working total is a judgement call, you may choose to always have enough power if every unit peaks at the same time, you may choose to average out but this will ultimately be down to how much money you have to spend in your monthly rental budget.

Once you have worked out exactly how much power you need you can start to ask data centres what they can provide. Typically they will provide x AMPS to a rack for £y per month. So as a comparison (real world figures btw) i have the following:

Which clearly shows data centre A as being cheaper than B but C is not as clear. This is something that sales people will rely on when talking to you and discussing their rates “ahh but we give you 22 AMPS for that price not 16!” well yes you will but how much does it cost me per AMP???

So now it is clear that Data Centre C is very expensive in comparison to the others.

In my case we wanted to put a blade centre into our racks and that thing can really consume the amps. We worked out that top load is around 22 amps so started to ask each of the data centres if they could accommodate this. What’s key here is that if they are not setup for providing the power to the rack then they are bodging it. It may well work but you are going to be pushing more power to the rack than it is designed to take which could well provide complications later (see cooling).

In terms of what to look for in a data centre you should be thinking in terms of two feeds into your rack from diverse feeds and SLAs against the provided power. Servers are often dual PSU so they can work off either feed and you would plug one PSU into the A feed and one into the B feed. The data centre might also recommend that you look at Power Distribution Unit (PDU) to plug in your equipment and monitor your power. It’s out of the scope of this blog but we very happily run APC PDUs which allow us to turn plugs on and off remotely and check how power we are drawing.  If the data centre can’t offer you practical advice on this or doesn’t tell you in too much detail how the power works then I would start to worry.

Although you could go into this in great detail it is worth remembering your Service Level Agreement. If your SLA says “100% power” then you can be relatively confident that they know what they are doing, and if they don’t then they are going to be paying you your money back! I wouldn’t accept anything other than 100% power in an SLA document from a data centre.

Cooling

We take this for granted really, a server room will be adequately cooled. How well is this actually managed though and is it environmentally sound? The data centre in question should be considering how much it will have to cool per rack based on how much power it is supplying to that rack. If a data centre is willing to put more power to a rack than is in the design specifications for the data centre (by say moving power from a nearby rack) alarm bells would start to ring for me. By doing so they create “hot-spots” where the heat being generated is too much for the cooling system to handle and as a result equipment overheats.

For the new generation of data centres there is a push to have them totally air-cooled by increasing the footprint of the entire data centre such that there are no hot-spots. Do remember that cooling is merely the conduction of heat from inside the building to outside in the most efficient way possible, too closely packed servers stress the system that is doing that work and increase the potential for hot-spots and overheating servers.

You should check the SLA document again and ensure its another 100% and start to ask questions on how they manage the cooling. Very en-vogue is the idea of using hot and cold aisles which work exactly how they sound - cool air to the front of the racks (where its get drawn into the servers) and then hot air where all the exhaust air is pumped out. Management of the floor space like this is a good indication that the data centre know what they are doing and have it under control.

Physical Security

This is up to you and how important your company data is. You have a number of things to consider here; how easy is it to get into the building and the suite in which your racks are located? (If you ever wander round Telehouse you will know what i mean) Do you need more security than just a rack? Do you need remote monitoring of your racks? Are there enough security cameras around the site? (in the UK - are you kidding!) Does everyone going and out of the building have to sign in and out?

Whether or not you need these are up to you but could have further financial implications to your rack rental cost.

You will need to be sure that the data centre has the appropriate ISO accreditation (27001 - UK only)  and that they practice what they preach! Some of this can only be down to experience; if someone is blase about letting you in when you visit then they are blase when you are not there and those are your servers in there. Of course the only way you can ever test the physical security of a site is by going and having a look and I would ask them to show you the security features whilst you are there.

A security cage is another good option to protect your servers; essentially a large metal cage surrounding your racks with some sort of coded entry system. Whilst this might sound like a good idea it occupies floor space which they data centre would otherwise use for more racks so if you want it your going to have to pay for it….

Ultimately, should the worst happen, you might have to reclaim on someones insurance and you should ensure that either you or the data centre has some to the appropriate level.

Fire suppression

It just has to be there! Hopefully there will be huge red canisters on the wall indicating a gas fire suppression system and it wont be something involving sprinklers!  If a data centre had a sprinkler system I would be out of there, I don’t care how good your early detection is if the fall back is to destroy all my servers then I just am not interested.

Ask the question, ask how it works and then think of the what ifs; What if a nearby room catches fire? What if my rack catches fire? What if someone burns some toast?

IP Transit

This is the second part of the data centre comparison matrix. How much is your data centre going to charge you to move your data around? You will want your networking guys involved in this as this is a complex subject which is not fully explored in this particular blog.

What this comes down to is how much data you think you will be transmitting at the same time and not the volume. As this will be most of your servers in the data centre you can check on the switches using SNMP in your currently location and find out how much they are pushing backwards and forwards. Typically there will be a single up-link into the switch that governs all your servers or, if there is not one, its probably a good idea to make one just for the purposes of gathering this data. Use a product capable of recording the information presented through SNMP by the switch (a product such as PRTG) and you can find out your average and peak data rates.

A data centre will sell you as much bandwidth as you need and are usually in a monopoly to provide IP transit (bandwidth) around their own facility as well as a peering point to connect you to the rest of the Internet. You could of course run your own cables between your racks and other carriers but if you need to do this then I doubt you are reading this blog!

So having retrieved the price for (in this example) 20MB/s CDR you can continue to fill in the comparison table:

The unavailable for data centre C was due to them wanting us to peer with a carrier rather than supply us themselves so it would not be a direct comparison.

You can see clearly that data centre A is expensive in bandwidth terms compared to data centre B and, if you are using  a lot of bandwidth, this could well be a problem.

It is worth pointing out that if you are going to be a high bandwidth user then you will have to think very carefully about your physical location before committing to a data centre. The further you move out of London the more expensive it becomes. This isn’t a problem for a usual company but where you are a media company (or have similar high requirements) the costs of bandwidth could eclipse your rack charges, as I have no experience of high bandwidth usages I will comment no further.

Physical redundancy of systems

Data centres are built to a specification or “tier”. Each tier is defined by the facilities which the data centre offers and the level of redundancy of those systems. Typical UK data centres offer a N+1 redundancy meaning that you need N number of air conditioning units/power units/etc to make everything work so you put in N with 1 spare meaning N+1.

The N+1 is an example of a tier 3 data centre. For tier 4 data centres everything becomes N+N, so every system has a standby version of itself - highly redundant.

The choice here is one of risk and expense. How much risk is your business willing to take on their systems potentially going down should there be a double failure in the data centre? If the answer is “not at all” then you might start looking for a tier 4 data centre. Be aware that there aren’t that many of these around so they will charge a bit more for their services.

For a fuller description on tiers and some more information on how data centers are operated the Wikipedia page does well: http://en.wikipedia.org/wiki/Data_center

Remote or Intelligent hands

Remote or Intelligent hands are often offered by the data centre to carry out tasks on your behalf in your absence. Typically these are things like powering a server on and off, changing a backup tape, installing a piece of hardware or plugging a cable in. The limit here is what you feel comfortable with and what the data centre feels comfortable with in both time and experience.

Of the data centre we use they offer 45 minutes per day of Intelligent hands for free. In order to use those we have to log a call and they undertake the task. There is often the question of “do we feel comfortable asking them to do this?” and sometimes its yes sometimes no. Obviously where it’s something we want one of our members of staff to do it costs us much more to send them.

You should be looking to design your setup with the view of never returning and using those remote hands as a last resort, but you need to check they are available otherwise you might come unstuck.

Managed Hosting

The other way you could go is managed hosting. In this model the data centre takes a much more active role with your servers and manages them on your behalf. This is an expensive but often great service where the hosting company will manage all aspects of your servers up to and including the applications they serve. An Exchange or a public facing website is a good example of why you might wish to use this service.If, for instance, you are a small company with a large on-line presence and do not have the IT staff in house to run a 24/7 hosting service then managed hosting might be the route for you.

Just check: 24/7 access

Might sound obvious but can you get to your servers night and day without any restrictions? You may take this for granted but its worth asking the question before you sign on the dotted line. What about deliveries of equipment? What about building your servers on-site, is there a build room? Practical questions might change your perception of your supplier and could give you an insight into the way they work and think.

Just check: Location Location Location

The last thing to check is where the data centre physically is and what kind of risks its location poses. This may sound like a “oh no terrorists” type comment but is about measuring up risk. Three particular scenarios spring to mind which you might just fire up google earth to have a look around before you make a site visit:

• Terrorist Activity
• Accident
• Force Majeure

Terrorist Activity

OK this is unlikely but why expose yourself to a potential risk? Do you need to be in Canary Wharf (surely a prime target for disruption of uk business)? If you do need to be there does everything need to be there or can you split it up?

Accident

Remember Buncefield anyone? Northgate’s UK HQ was around 50metres away from the oil storage depot when it exploded. This more or less destroyed the building (see photos). Is your data centre next to something that could go bang? (For more information on the Buncefield disaster the official investigation site has a wealth of information)

Force Majeure

Are you on a flood plain? Believe me this is something i checked! Sounds daft but you bet the Vodafone data centre guys in Turkey have this on their list now:

Data Center security camera recording - Vodafone Istanbul, Turkey

Due Diligence

This is the same when working with any supplier for the first time:

• References
• Financial reports
• Director checks
• Credit rating
• Contract negotiation

These should be pretty standard so I’ll leave these alone.

Final thought - which one do you choose?

There are still other technical aspects of a data center decision that each IT leader must choose for themselves.  Most revolve around the kind of environment required by the business and what kind of IT environment that you want to deploy.  For a large SME (of which I currently work for) the choice is clear; get out of London and exploit the lower cost of data centres outside the M25 and the rapidly decreasing cost of bandwidth to get the most for your money.

Oh and if your interested in seeing how a data centre is built have a look here: http://blog.gyron.net/