The Winding Path to Xsan Nirvana

The Apple (Nasdaq: AAPL) Xsan system follows the relatively common path that other SAN (storage area network) products have established, which is to simplify storage management, and the Xsan delivers -- but it takes a lot of planning to reach Xsan Nirvana.

An overview of the Xsan system requires technical and budget planning. Since its initial release, the Xsan has been hailed as a "high-performance enterprise-class SAN solution, priced at (US)$999." This is for the Xsan software only; the other components to make a complete system are more. All things being equal, here's the middle of the road cost for a 14TB system:

Compare that with the comparison CapitalHead.com did last year between the base storage costs of a direct-attached storage (DAS) and an entry-level SAN system. The installed costs of the Xsan are only $1/GB higher. Top-of-the-line enterprise storage can go as high as $6/GB, so the Xsan is fitted nicely above entry-level SAN systems.

Setting It Up Right the First Time

As with all successful server installations, planning is required before deploying the equipment. Planning in this case means more than finding the best cost for hardware. Planning involves estimating your client's needs, knowing where the equipment is going to be sitting, configuring the network aspect of the Xsan, configuring the storage, setting up fibre channel fabric and port settings, and determining the access and permissions (Open Directory) method.

According to Aaron Freimark, the IT Director of Tekserve, an authorized Apple reseller and service provider, and also the founder of Xsanity.com, a forum and blog Web site for the Xsan IT community, the top three Xsan setup missteps are the following:

• Incorrect Domain Name System (DNS) configuration
• Inadequate cooling and cabling requirements
• Overly conservative estimates of storage requirements

The book Apple Training Series Xsan 2 Administration: A Guide to Designing, Deploying, and Maintaining Xsan by Robert Kite is an excellent resource for your planning. At the beginning of chapter two, this warning stands out: "It's easy to add storage to an existing Xsan SAN, but reorganizing a SAN after you've set it up is not so simple. Therefore, it's important to plan the layout and organization of your SAN and its storage in advance." The same planning advice is required for the Fibre Channel and Ethernet infrastructures.

The underestimation of storage is a common problem reported by experts. Graham McGuiness, codirector of Root6, supplier, and integrator of broadcasting and film technology, says that having a realistic view of storage needs during the planning phase is a challenge. "Today's projects are under budget pressure," McGuiness says, "yet clients that are trying to deploy just the amount of storage they need now, quickly find out that they need more." Fortunately, Xsan storage is scalable, and adding additional terabytes of storage is easy.

Whatever the final environment looks like, there should be good documentation and a backup plan. Herein lies a challenge: Some Xsan installations have so much data that they cannot back it up. Here's where you may need to change hats from IT administrator to IT continuity planner. Richard Frasier, Apple certified system administrator and technician for Loop Enterprise, a managed IT service provider, says, "making a plan in case of failure rather than waiting for a failure to happen" is often a missed item during planning and deployment.

Xsan IT and Storage Management

A common misconception of users is that when a new server is purchased, the IT department unpacks the equipment, puts it in the corner, plugs it in and it just works. Any IT admin will tell you that this is just not the case no matter what the product is. Freimark, quoted earlier, says that the Xsan environment is a completely different world. "You have this organically linked network of computers; what happens to one [system], happens to everyone else. It's really important to pay attention to the whole network that you're creating. Because you're dealing with the network as a unit instead of a group of machines, it will expose any sort of problems your network has."

Incorrect configuration of the FC switch, Open Directory, or DNS can result in latency issues that make it appear that there is a bandwidth problem. Freimark recommends that if more than one FC switch is in place, then the IT staff should research "Domain Locking." This assigns the order of switches and defines who is first and second and so on, as well as the traffic being assigned to respective LUNs (logical unit number). If domain locking is not turned on and a reboot of the FC switch occurs, the LUNs won't be available. That can be a scary moment for an IT admin after firmware or software updates are applied and the LUNs do not show up.

The data volumes you create should have a documented naming convention and the more straightforward your storage configuration, the better off you'll be. It is also best practice to back up your "/Library/Filesystems/Xsan/config" folder. Freimark says this is the key to your SAN. The information in this folder could be rebuilt by hand, but you don't want to go through it. There was an example of this on the Apple discussion boards in 2007. Help from the Xsan community paid off, and the IT admin was able to discover and associate his LUNs and disks by their serial numbers and rebuild the Xsan config files. The effort was successful, but it was clearly a stressful experience.

Xsan logs can get very verbose and detailed. During a troubleshooting exercise, it may be easy to lose sight of what the real issues are. There are no easy shortcuts here. The logs tell you quite a bit, yet you'll have to determine through use and experience what is important and what is not. This is where the Xsan community comes in handy. By reviewing posted logs and how others have interpreted them, you will gain insight into analyzing your own logs.

Don't feel awkward about posting your own logs to get feedback from them other experienced IT admins. Additionally, if you discover something in your Xsan logs and are able to determine a work-around or maintenance item to watch for, post it on Xsanity or Apple discussion boards. The collective archive of knowledge can help someone else.

Professional Experience Required

The Apple Xsan was developed for high performance -- from the operating system to the file system, the SAN is built for speed. Apple Xsan deployments from the user perspective also require planning. Finally, helping Xsan users learn how use the system is an important project management step for a successful deployment.

Root6's McGuiness comments that users who used to save their work to their workstations now have to think differently -- they have to think in terms of a shared environment. "Users do not need training," McGuiness says, "just a little assistance to adapt to the new system." Once users get it, they can't see how they worked before without an Apple Xsan, according to McGuiness.