Tag Archives: EMC

Provisioning EMC Storage for Windows 4X Faster with ESI

The EMC Storage Integrator for Windows was made to simplify and automate many of the mundane tasks associated with provisioning storage in a Windows environment. It is a free download and comes complete with a simple MMC interface as well as PowerShell commandlets, SharePoint provisioning wizards, and System Center plugins for Ops Manager and Orchestrator. Good stuff here.

Advertisements

EMC DMX w/VP as Storage Utility for MSIT SQL

If you are wondering how SQL might work in a virtual environment, consider the following paper from Microsoft.

Green IT in Practice: SQL Server Consolidation in Microsoft IT

To sum it up, SQL has been thought (incorrectly) to be one of the few databases that are best kept on physical, but here Microsoft IT has displayed a sensible approach to identifying and consolidating the right virtualization candidates, through a process they call RightSizing. The Microsoft IT application portfolio consists of about 2,700 applications. There are approximately 100,000 databases on 5,000 SQL Server Instances, most of which are on dedicated hosts. Approximately 20 percent of those hosts reach end-of-life each year and are replaced. Average CPU utilization across these hosts is below 10 percent, indicating significant opportunity for host consolidation.

Microsoft IT developed the RightSizing initiative to ensure effective utilization of servers in the data center and in managed labs. Because significant underutilization occurs, one of the initiative’s first tasks was for Microsoft IT to identify underutilized servers that might be good candidates for virtualization (for more information on RightSizing, see Resources). The Capacity Management team relies on RightSizing data.

To accurately compare the performance of server platforms of varying architectures, Microsoft IT has developed a Compute Unit (CU) formula for each server platform that utilizes an industry standard, architecture-agnostic, benchmark suite from the Standard Performance Evaluation Corporation (SPEC). The SPEC benchmarks are developed in such a way to allow a repeatable test with strict result submission requirements. The Microsoft IT CU formula uses a baseline (not peak) benchmark that measures the rate of integer calculation work a server platform can perform in a given amount of time.

They compare a lot of before and after scenario’s in the paper.

  • “Before” is mostly an app on a server with direct attached storage.
  • “After” is the new storage utility based model.

The new model places all of these SQL-based app’s atop an EMC DMX-based storage utility that dramatically raises the performance, scalability, and availability needs they had previously offered their users.   They use the DMX Virtual Provisioning (our implementation of thin provisioning) to preallocate (400) 300GB drives and eliminate the need for host additions and re-zoning and all that junk. It’s a perfect use case for DMX VP.  I attached a whitepaper on this at the end of this post.

Some highlights from the paper:

Their storage utility and virtualization approach, in comparison to their older DAS/legacy environment, is quite GREEN: it saves power and money.

legacy-vs-new

Some performance tips for virtualizing SQL with Hyper-V:

  1. Crawl, walk, run. Don’t go for the maximum host consolidation ratio right away. Begin with the smaller workloads, validate your deployment, and refine the plan. Maximize your resource utilization in phases after establishing and evaluating your actual utilization.
  2. Use Hyper-V pass-through disk or fixed Virtual Hard Disks (VHDs) for storage (Figure 6). Fixed VHDs offer some manageability benefits but provide slightly lower performance. Moving a guest to another host, for example, is simplified when using VHDs.
  3. Do not overcommit processors for SQL Server guests. Begin with one logical processor for one physical processor. Verify your performance and refine your configuration as needed. At some point, this may include overcommitment, but begin without overcommitment to manage risks.
  4. Avoid the use of emulated devices in Hyper-V. Favor synthetic devices which provide better performance and lower processor overhead.
  5. Establish an operating level agreement (OLA) with performance requirements for your storage provider if they are a separate service. Microsoft IT SQL Server Utility has requirements for 1ms average disk/second read/write for log and 8ms for OLTP data.msit-storage-utility

Their Approach (build a solid foundation):

The Storage Utility service provides shared or dedicated SAN storage to which data center servers can connect. The service provides the SAN storage and all hardware required to connect a server to the SAN, as well as all maintenance and management functions, but does not provide the server itself.

An effect of the Storage Utility was that instead of engineering and purchasing small, medium, large servers with anticipated DAS for SQL Server and other server types, Microsoft IT was able to modify the standard builds so that minimal DAS was included in the purchase. This means that application support teams no longer had to anticipate disk capacity over the life of the server which usually resulted in underutilization.

Their Storage Utility Configuration:

san-sql-utility

You can learn more about EMC DMX Virtual Provisioning in this paper:

Implementing Virtual Provisioning on EMC Symmetrix DMX with Microsoft SQL Server 2005

A customer had a SharePoint farm. Eee-i Eee-i O.

And on this farm they had a 2007 upgrade that needed to happen.  After a Microsoft Consulting Services review, they were told to fix up a number of issues in their current environment:

  • Get web/app/DB servers  in the right place.
  • Intranet and internet portals on shared disk spindles, they want to move to same hardware (SAN) w/dedicated spindles.
  • One content DB was 210 GB.  That DB needs to lose some weight.
  • They had 40,000 resources in a single resource group… Microsoft said they needed to change this (db size and resource #’s) because they were outside supportable boundaries. Basically, this becomes a real bear on the system, and it’s really difficult to backup that many files.

They are using a DMX3 Tier1 146GB 10K for content DB.  A DMX4 is being purchased for this environment and they will share this will other applications in the company.  No problems with the storage.

For backup they use the point product AvePoint for MOSS backup.  They use a multi-phase process to push Dev to Test to Cert to Production. Seemed OK, but then they thought a second and said.

“You know what, there is an opportunity for a better backup product to push data through the dev to production cycle.”

I asked more about this and found, like many customers, they deploy MOSS farms in stages.  The current stages of their MOSS lifecycle were:

1. Dev – Virtual everything (DB, Web, App)
2. Test – Virtual everything (DB, Web, App)
3. Cert – Virtual web, app, physical DB (same as prod)\
4. Prod – Virtual web, app, physical DB

Since EMC is known for it’s proficiency in virtualization and data protection, they thought we might have a slick product (like Replication Manager or Networker) that could assist in the movement of data along these physical and virtual tiers without having to backup and restore every time.   Well, yes we do.

Well, I told them, it’s great timing. The Networker team has not only made the product much simpler to use – the Fast Start edition of the product can be installed in under 20 minutes – they also have been working really hard on their SharePoint support.  The new Networker Module for Microsoft Applications (NMM) they offer the trifecta of Sharepoint backup and restore functionality:  network backups, VSS application consistent backups, and fine grained item-level restores.  Whitepaper here (or click the picture below).

LAN Free SharePoint Backup/Restore with Networker

LAN Free SharePoint Backup/Restore with Networker

Next step: the customer is going to test this with EMC’s Proven Solutions group, where EMC works with Microsoft to demonstrate our technologies working together.  For SharePoint, we test large and small farms in physical and virtual server environments, load them up, and see where they break.  The people doing the testing are having a blast doing cool stuff every day to push the limits of datacenter technology.  Sometimes the work they do is more Mythbusters (isn’t performance worse on virtual? BUSTED), and sometimes it’s more like CSI, which usually starts with a dead body and then a lot of time is spent trying to see what the hell happened 🙂

I do know each day is different and fun for them, so the resulting documents you see are extremely good. Unfortunately about half of it is buried behind EMC’s internet portal PowerLink, but that is being fixed…

Here’s a sample of what is available publicly at the moment:

Consolidated versus Distributed

I just got back from Australia visiting several large EMC customers (saw Meg Ryan randomly when we were there). Even down under, the whole Exchange 2007 sponsored “SAN versus DAS” debate is still alive and kicking…

Logically speaking, isn’t it easier just to share and consolidate stuff?

Examples of “stuff”:

  1. Airplane video monitors. More monitors, more problems.  Central systems are more efficient.  First class passengers, sure they get all the video/movies/games they want.  But they pay heavily for the convenience and additional control and of course for the additional maintenance.
  2. Bank accounts. Isn’t it easier to do your banking from one place instead of having different accounts for your stocks, mutual funds, savings, checking, money market, and mortgage loans?  I’ve finally got my finances down to two places…
  3. Boats.  Those who know say it’s much cheaper to chip in or rent than to own your own boat.  Renting also lets you try out different “tiers” of boat – power, sail, speed, cruisers, etc.
  4. Libraries. “Networked libraries” where you reserve a book and have it sent to you are substantially better than searching or driving to each library one by one to find the book you want.  I’m a beneficiary of a networked library (part of the minuteman library network). It’s awesome.
  5. Debt and student loans.  You can group your liabilities and get a reduced rate from the hopefully not-in-default financing organization that wants the interest $ from you, or you the old way, you deal with each bill one by one.
  6. Companies/Government Agencies.  Organizations (including companies/community groups/towns) are frequently consolidating to be able to provide shared services for one another.  Hamilton and Wenham Massachusetts are actually discussing a merger to be able to lower costs by utilizing shared resources such as schools, rec centers, libraries, and potentially fire and police stations. They are already at a stage called regionalization (which is the direction towards consolidation).
  7. There’s probably more, but I am out of ideas at the moment.

Here’s another statement from the Town of West Boylston (again Massachusetts)

Regionalization allows communities to share administrative and program costs between two or more communities in order to increase or maintain the level of services within the participating communities.  I often wondered why every community needed its own police department, its own fire department and its own library.  I find it hard to believe that this system will exist 15-20 years from now as municipalities struggle every year to provide even basic services to its residents.  One of the areas where Massachusetts has some success with regionalization is with regional school districts.  Regional school districts allow communities to share costs and to provide services they could not provide if they were acting on their own.

More reading if this is not putting you to sleep:

Centralized vs Dencentralized IS Organizations in Companies

Higher TCO when Decentralized

Records Management Tip – Centralized is Better

Tips for Dealing with Your Storage Admin

It happens everyday.  Yet another SQL admin falls victim to the technical prejudice of those who provide their storage.  It’s a sad story which is seen everyday throughout the currently developed world; at least those working in relatively well-funded companies.  UNIX guys make fun of Windows people. Mainframe guys refuse to believe Windows is even available.  Oracle guys say their DB is the king. The reality is that Window servers are here to stay in today’s datacenter – whether it’s in house or in the cloud is a story for the future.

My point is to reach out to all you application owners, and although I’m more of a storage guy, I do know a bit about applications and getting them to perform better.  I know more about how companies bicker internally after visiting many, many customers with EMC.  I give workshops to SQL, Exchange, and SharePoint app owners, and especially when the storage guy is sitting in the room, it seems like a therapy session.

Bad Performance = Bad Code or a Bad Config?

When it comes to SQL performance troubleshooting, a friend of mine always says, “it’s either bad code, or a bad config.” This is not entirely true – there are so many more things which can cause slow performance on a server – but it’s a reality of the situation which typically happens. The storage team says it’s something with SQL. The SQL admins revolt and demand it’s not their fault, and to see how the heck their storage was setup.  These two groups are often at war with each other!

So, my friends, I come in peace as a friendly mediator, and I’d offer you these tips:

  • Befriend a storage person. If you have to request storage from a storage team or a person, you should identify these people, seek them out, and take them out to lunch or something – it will make life a bit easier.
  • Learn about storage a little bit. It’s not all that hard, and the variables are outlined quite simply. Using these words can help you make a proper request for a storage allocation for your servers which of course are running your applications and databases.  If you need more, I’d suggest starting with a nice site called SearchStorage, which is part of a larger IT web network.
  • Before deploying anything on EMC gear, assume we have a piece of documentation which outlines the steps necessary to put SQL (or Exchange or SharePoint or whatever) on one of our SANs.  If we don’t; well, shame on us – and you should push a bit to see if we can whip something up quickly for you.  I am not saying RTFM, I am simply stating that we at least owe you the M part.
  • Ask about application integration of the in-house storage vendor. Now that you are buddies with someone on the storage team, sit down and ask them about the cool stuff that each storage platform that they run can provide for you like 1) app-consistent snapshots to slice your backup windows down to nothing or 2) app-consistent replication to make sure your app comes up with minimal pain (or data loss) after an outage.
  • Wait a Virtual Second. Well, if you are like the rest of the business world, you are probably looking to virtualize some percentage  of, or specific servers/applications in your IT environment.  If the storage team has done this, you have to make sure you understand a bit about that too because for better or for worse – it changes everything.
  • Get them to build a storage request form. Many companies are already doing this. Some as simple as your name, your email, your capacity, and your IOPS.  This is enough for most situations, but I would add RAID-type, latency, and LUN sizes if at all possible.  If you’ve done all of the above, and taken an appropriate performance measure from the items you are able to monitor, then you should know what specifics should be included in your storage request form (and of course who it goes to).

The more technical version of this needs to make it’s way onto this site soon, and that would have to include a quick review of the performance chain in the SAN world, and how to find the weakest link.  Stay tuned.

EMC’s Windows Related Webcasts Available On Demand

For a while EMC has been running their own internal web events to allow the product teams and engineers to communicate directly with people who might become customers.  Below is a list of several events that you might find interesting that are available OUTSIDE the firewall.  Enjoy!

Exchange

Microsoft Exchange 2007: A Second Look

Leverage the Benefits of IP Storage for Your Microsoft Exchange 2007 Infrastructure

Best Practices for Deploying Microsoft Exchange 2007 in a Virtualized Environment

SQL

How Celerra snapshots can be used for a mission critical SQL server environment

EMC Replication Solutions for CLARiiON Decision Support (in a SQL environment)

Virtualize Mission-Critical Microsoft SharePoint and SQL Server Environments with VMware and EMC

SharePoint

Networker: the most advanced SharePoint backup solution on the market

Virtualize Mission-Critical Microsoft SharePoint and SQL Server Environments with VMware and EMC

EMC Documentum Connector for Microsoft Office SharePoint Server 2007

Managing Global Energy Projects with EMC Documentum and Microsoft SharePoint: A Customer Success Story with TietoEnator