Exercise due diligence when leveraging new technology for business transformation

Technology advances continue at unprecedented clip levels. Keeping up is a challenge. Being able to truly use the embedded features and functional capability of information technology products is rarely fully exploited.

The limiting factors include allocating enough time for education, learning the given technology realm fully, and developing best practices and processes that pave the way to success. Take IBM's SAN Volume Controller (SVC) as an example.

The SVC, properly designed and implemented, greatly increases utilization. The benefits include having to buy less capacity and/or delaying the need to add capacity (due to the higher utilization factor), multi-brand storage device interoperability, improved manageability, and less downtime in performing routine maintenance and migrations. Huge benefits.

How can organizations realize these benefits?

The answer lies in changing the thinking and approach to storage management. IT managers must make an initial investment in learning what it really takes, in the form of education and process definition, so that sys admin staff can extract the full potential in these benefits. SANs were a great step in the right direction, but virtualizing the storage environment is a new ball game. Best practices must be correctly ascertained and put into true execution. A suggestion is to set goals regarding utilization, workload management, and new levels/targets for up-time, then reward the staff when such goals are achieved.

The capabilities of new technologies available today are extensive and offer huge improvements to existing environments. Almost all have fully improved from the habits and practices of 2001 or prior. Virtual storage, virtual I/O, and virtual servers are just some examples of new realms that can provide benefits, but only if organizations adapt and adopt new philosophies and approaches to learning, designing and managing solutions with an emphasis on best practices.

Investing in "capable technology" optional features can be a total waste, or will be for naught, if the commitment to learning and the willingness to change the ways an organization manages these new resources are not in place. The benefits far outweigh the costs involved for companies of a certain capacity/size realm or higher. Demand that your alliance partner analyzes your situation and applies a design that hits the mark for you. You need an individualized plan with a granular explanation of the financial return that delivers an obvious and compelling reason to move ahead.

Executive summary
The cost per megabyte of storage is going down at record rates. So, how can it be that the total cost of storage is going up? Why are some large organizations reporting that storage costs have risen to be 55% of their total IT spending?

The answers to this conundrum lie in the following areas:

• An insatiable appetite for storage is resulting in compound growth rates of capacity that overwhelm the cost reductions in the technology.

• Fractured organization of the storage assets results in low utilization. Users are forced to buy more even though what they have is underutilized.

• Rapid growth leads to complexity which in turn leads to high administration and management costs. In extreme cases, every $1 spent on hardware costs $8 to manage, making management the key factor in total cost of ownership.

The following articles explain the background to these trends and identify some of the key directions in reducing the total cost of storage ownership.

Storage - the big picture

An overview of trends, challenges, solutions.
An introduction to the other articles

Trend: Mind-boggling demand for capacity

It used to cost a lot of money to store data on computer disks. That is no longer true. Sometime in the mid 90s the disk cost blasted through the paper and film barriers and disk became the cheapest way to store most forms of data. Since then, disk storage cost-per-megabyte has continued to decline at a compounded rate of over 50% per year. Little wonder then that the demand for disk storage continues to explode.

The "man-in-the-street" gets confused at this point. "Where can all this data possibly come from?" To those of us that deal with this every day, the question seems ridiculous, but it is worth reflecting on just some of the reasons for this phenomenal demand for storage:

• Archives: Organizations usually want to keep (or are required by regulations to keep) most of the digital data they have ever created. So, new operational data needs new capacity to hold it.

• New media: Consider a professional user who has been creating text documents for years. Once that user starts to put photographs in reports, s/he can use more storage in a day than they previously used in a month. The first "document" they use that includes an embedded video might require more storage than all the previous text documents they produced in their career!

The amazing truth is that the demand for digital storage is growing faster than the unit cost of storage is going down. This fact alone would result in storage getting a higher portion of IT budgets.

Sprawl and disintegration

The 90s was an exciting time; the Internet, Y2K, the Internet, inexpensive servers, the Internet, broadband for everyone and, of course, the Internet. But the 90s was definitely not the golden age of system architecture. In retrospect there was a scramble to spread the new technologies. IT disciplines established in previous decades were often casualties to the need for speed. Organizations that for two decades had managed to run efficiently with one or two "computers" suddenly acquired eight or twelve "servers".

As servers proliferated storage tended to be dragged with them. Previous progress towards storage pooling was often reversed as servers popped up like mushrooms with their own lump of dedicated, directly attached storage.

Challenge: Consolidate and integrate

In the new millennium the word "consolidation" entered IT jargon. Much of the recent history of IT can only be understood in the context of the consolidation that was necessitated by the previous explosion of fragmented growth.

Storage pains

Multiply together the effects of massive capacity demand and the need for consolidation and you find the pains that are commonly associated with storage over recent years:

• Low utilization. A sprawl of direct-attached, isolated storage subsystems only attains good utilization by accident. IT managers began to complain about average storage utilization below 40%. Their CFOs complained more loudly at the requests to buy more storage when most of the current assets were not being used.

• High management costs. Variety and complexity led to soaring administration and management costs. There has been a big outbreak of irony as the very people who are bringing automation to their organizations don't have the automated tools to perform their own functions! As a result, the analysts' studies started to show alarming numbers like "for every $1 spent on storage hardware it costs $8 to manage it".

• Availability. Storage is one of the three legs of any system; without it the system falls over. Yet many people were, or still are, living with storage systems that are not built for high availability. Despite the common appearance of hot-swappable components, many files, databases, or even whole disk subsystems cannot provide continuous availability during normal daily processes such as backup, reorganization, deploying new hardware, or moving data to a different disk.

• Untenable budget demands. This is the natural result of the other storage pains. As IT organizations try to meet the exploding demand for more capacity they see the storage portion of their budget grow faster than the rest. Several studies started to show 40% of total budget early in this decade with some claiming 55% and still growing.

The tantalizing truth is that many people haven't been able to reap the benefit of the plunging cost of storage. On the other hand, the benefits of Moore's law in servers, network infrastructure and bandwidth seem to be reaching the bottom line of the IT budget. Thus, for many individual users and for the industry that serves them an imperative has been created; find the breakthroughs in storage that will remove the pains and deliver much more of the potential power as real benefit.

Solutions - short term

Where should we look for the breakthroughs? It should be clear by now that we shouldn't look for solutions in faster or cheaper hard disk drives. It looks as if we can go on expecting those to be delivered 50% cheaper every year. But, just as that hasn't removed the pains over the last few years there is no reason to expect it to remove them over the next few.

In the short-term we should look for solutions in two areas:

• How we organize the basic technology building blocks into an integrated storage subsystem.

  • Pooling and sharing are obvious ways to cure low utilization. Integration through system architecture has always been a powerful way to reduce complexity and create efficiency. A powerful enough architecture can encompass all functions, including backup, recovery, change and growth - leading to a storage subsystem that is continuously available by design. Most of these ideas can be encapsulated in one word; network. Reorganizing current isolated storage assets into one (or a small number) of shared networks is a key part of any important solution.

  • We are also fortunate to have another weapon in the "better organization" armory. It's ability to transform and integrate can sometimes have an almost magical power. It is virtualization. Starting with virtual memory almost 50 years ago, virtualization has been solving some of the trickiest problems of utilization, integration and consolidation. Once again in storage, it is showing its power.

• How we manage the storage networks. You could say that administration and management facilities are inextricably linked to the architecture and design of the storage network and can't be separated from the above. If you did, you would be right. However, from a pragmatic point of view, most systems management solutions are delivered as software and often are evaluated and acquired separately. Once you bake a cake, you can't get the eggs back out again; but if you want to bake the best cake it may help to know that chickens are essential. Systems management software is essential to our short-term storage solutions. Even though, in theory, it shouldn't be treated separately, in practice you may have to - just as we do here and in our more detail articles that follow.

To summarize the short-term solutions, they consist of a combination of storage networking, virtualization and systems management software. These can be very effective storage pain killers and, in many cases, create changes worthy of the name - breakthrough. Why are we so confident in making this statement? Because, rather than a prediction it is a statement of history. So far this article has simplified reality as if everyone has these problems today and the solutions are only in the future. The truth, of course is more complex. Many people suffer these storage pains today and some, mostly small organizations, are pain free (often in the process of creating their own future doses of storage pain). But there are many organizations that have suffered and created their own cure. It is as if these organizations carried out the clinical trials and have shown the efficacy of the solutions we now confidently prescribe in the following articles on storage networking and storage management software.

Solutions - long term

In the fast-paced world of IT, short-term solutions are inadequate to ensure success - unless they are part of a long-term vision. A vision that goes beyond the current challenges and is broader than the subsystems under study. In storage that vision is Information Lifecycle Management (ILM).

Think about the most important goals we set ourselves; zero-defect-quality, being the best in our industry and so on. These are never objectives we achieve and then we're finished; rather they are continuous journeys that have no end. So, it should not surprise us that ILM is like that too. It is not something you "install" and you're done. Rather it is a journey where you have a clear vision of the destination but you expect to keep traveling for a long time - maybe forever. Like the other goals mentioned, the only time to start the journey is now. Some pundits are saying "Many organizations should wait until ILM matures". That's as nonsensical as saying "Don't try to improve your quality until someone else has developed a complete quality program for you." Those organizations who implement their short-term solutions in the context of a stated ILM policy will move in the right direction - albeit slowly at first. Those who listen to the pundits and "put it off" for three years will find themselves well behind.

Notice that we haven't even tried to define ILM. This is one of those rare IT terms that is self-explanatory and common sense - another indicator that this concept is probably superior to many others. So, the name is sufficient description for this overview; in the ILM article below we will dissect it further.

Conclusion

The importance of storage in IT has grown and will continue to grow. Storage is like an adolescent going through a growth spurt - it is experiencing growing pains. The immediate solution to these pains is a judicious mixture of storage networking and storage-related systems management software. However, our storage adolescent is only likely to grow into a successful, mature adult if we apply these short-term remedies in the context of an ILM strategy.

The following articles take this bold prescription and examine it in enough detail - attempting, as we go to indicate the best practices that will result in success.

... click for a Webinar replay on one aspect of storage management

Storage networks

Why are networks "the solution"? How to choose between NAS and SAN?

Pooling capacity, sharing resources and providing any-to-any access; these sound like good ideas in any endeavor - and these are the general benefits of a network approach. Therefore, it is no surprise that storage networks represent the basic direction of the solution to storage problems. But there is a surprise - fortunately a pleasant one - in that storage networks provide impressive benefits in other areas, such as backup and recovery.

So, networks provide impressive functional solutions. But total cost of ownership is increasingly about management costs. Surely networks are more complex and could increase management costs? It all depends on the completeness of the solution. A functionally rich network with poor management facilities could be costly to manage. A solution with integrated, complete management tools can dramatically reduce management costs.

The next article will focus on management. In this article we focus on the functions of storage networks.

NAS and SAN

First, let's understand the two main types of storage network. The obvious starting point is a definition; but if you surf the Web for a few NAS/SAN definitions you will quickly discover that, in this area, definitions don't get you very far! To save you time, here is one of the classic definitions that tries to differentiate the two types of networks:

Network Attached Storage: a storage subsystem that allows multiple servers to share files.

Storage Area Network: a storage subsystem that allows multiple servers to access shared data at the block level.

This definition is valid - just as are all the others. But it is not very useful for a real-world practitioner trying to decide which network to implement. Later in this article we will explain why most attempts at more meaningful definitions fail.

A more fruitful approach to understanding the real-world differences in these techniques comes from understanding their history. So, let's look at each one in turn.

NAS - a short biography

In the early 80s file sharing was business as usual in the mainframe world, but it was a brave new world for the LANs being created with UNIX "minis" and the white-hot PCs. The UNIX and PC file sharing standards introduced in 1984 have proved to be resilient and are still with us today. They enabled the era of file servers that lasted almost two decades. A file server was nothing more than a full server running the same operating system as the servers that ran the main applications. They worked, but with problems:

• End users were not impressed with file access speeds noticeably slower than their local drives.

• As more organizations embraced mixed UNIX and Windows environments (not to mention mainframes, iSeries, Linux, ...) the mandatory separation of Windows and UNIX file serving was underwhelming.

• These were general purpose servers recruited for a specific purpose. So, much of the management was manual. Allocating capacity, managing capacity increases, managing network connection and performance, administering backup and recovery - these proved to be increasingly complex and costly.

When NAS filers arrived they attacked these problems logically and directly. Looking back, the NAS solution seems so obvious that it elicits the "I knew that" approach in most of the pundits. Here is how NAS filers solved the problems:

• Specialized the hardware and operating system to the specialized task of file serving. If you examine in detail how a standard UNIX or Windows OS handles  the common functions in file serving it will make you smile; the number of interrupts and instructions involved is almost amusing - providing you are not paying for them! By changing the architecture and rewriting the key parts of the OS, the NAS providers were able to slash the instruction path lengths of the functions that a file server performs millions of times a day and eliminate all sorts of functions that are needed in a generalized OS and never used by a filer!

• Integrated both NFS (UNIX) and CIFS (Windows) protocols into a single product. Now you could serve files to all servers/clients on your LAN. You could even serve the same file to both camps as if it came from their "native" server.

• Designed products with an appliance mentality. The "appliance" approach can be effective in many areas and there is no special reason why it should be tied to file serving. Perhaps it is just an historical coincidence that NAS filers were the first major IT product to bring this approach into the mainstream. In any case, it was very effective. Design the box from the ground up to be easy to install, administer, upgrade, maintain. This provided a direct assault on the complexity and cost of "traditional" file servers.

So, NAS filers ruled the world? Not quite. They had some limitations that were better addressed by an alternative ...

SAN - a short biography

As you can see, all these stories start in 1984! Perhaps George Orwell had a point after all? SCSI was merely a protocol to allow servers to request disk drives to perform read and write operations for specific blocks of data. It started life with a limited purpose - make it easy to use standard hard-disk drives on any small computer. So, we shouldn't be too harsh when we look at the list of severe limitations in the initial specification.

Struggling to improve SCSI the engineers realized that all of the limitations were in the physical end of the SCSI protocol. Unfortunately the creators of SCSI had ignored the experience of network protocols (like SNA) and had created a monolithic standard. However, such is the power of layered architecture they were able to impose it retroactively and SCSI-3 was the result. As the chart to the right shows, the layered architecture allowed different implementations at the physical and transport level. As a direct result, SANs could now became a practical reality.

Fibre channel was initially the dominant implementation for the transport layers. Since this was relatively expensive, initial implementations were limited to big, expensive networks, leading to the misguided reputation that SANs are inherently costly and complex. However, the architecture contained the potential to blow that reputation out of the water - but more of that later.

Architecturally, a network of servers and storage devices looks and operates remarkably like a network of servers and clients on a LAN. So, the name SAN - Storage Area Network - was born for such a network using SCSI protocols to share storage devices.

The architecture of SCSI-3 is important to our story, so we will return to the architecture diagram  in the "Future of SAN" section where we will continue this discussion.

NAS versus SAN

So, now we know what they are, the obvious next question is "Which is better?" Unfortunately, we have no answer to that general question. There are situations where either wins.

I'm confused!

Why don't we use the name SAN for a  LAN with servers and storage devices sharing data? They look the same.

Join the club! A lot of people get confused, and yes, they can look remarkably similar topologically and functionally. In fact it is amusing to read the pundits as they tie themselves in knots trying to explain how a SAN is fundamentally different from other storage networks. For example, some people have built sub networks that use LAN technology for the dedicated purpose of handling storage traffic. Surely this is a storage area network? "No" say the pundits. This is a SLAN - a storage LAN.

The truth is that SAN is arbitrarily defined as a network that shares storage devices using the SCSI protocol. It's that simple. Attempts to "understand" the principles behind a term that is, in fact, not based on principles is what leads to the ineffective mental gymnastics.

The next obvious question is "In that case, what types of situation favors one or the other?" We are even going to duck this one. The factors that affect your real world choice are so numerous that rules-of-thumb are likely to be less than useful - where "less than useful" is a euphemism for wrong!

We will illustrate this by reviewing some of the conventional rules-of-thumb that you can still find in magazine articles and analysts' reports. We hope this will show why we refrain from putting our reputation on these oversimplifications.

Conventional wisdom

Comments

NAS only does files. It isn't suitable for databases.

There are many databases operating successfully on NAS. Some big and complex ones.

SANs only do low-level block sharing whereas NAS does high-level file sharing

There are SAN File Systems that do file sharing out of the box.

SANs are more powerful and faster

NAS filers are often fast enough for the most demanding needs. Thus the maximum speed difference may have no practical significance.

NAS is easier to understand and manage

These have been relative strengths of NAS. But SANs are no longer exotic; they constantly get simpler and lower-cost.  See the "Future of SANs" to see why any remaining differences could disappear.

NAS is less expensive

XXX is better for backup and availability

Depending on which manufacturer's Website you go to you can replace XXX with SAN or NAS. Perhaps the only solid claim is that SAN is better at sharing tape drives across the network.

Why is it so difficult to give simple generalizations? The answer is clear. In the early days of these networks, several of the above conventional wisdoms were valid. But NAS and SAN solutions are being constantly improved by some very clever people. Where a real advantage lay on one side or the other, the "loser" used ingenuity to reduce, eliminate or even reverse it. As a result, you have two very powerful, mature alternatives. With this good news comes the bad news - choosing is not so easy any more!

In addition to your functional needs, the "right" choice for you will depend heavily on your history; your current inventory of skills and your installed inventory of storage devices.

Just to add a final nail in the coffin of easy choice, combinations of NAS and SAN are now no longer rare. For example, early NAS appliances used to have everything in one box; the server (processors, OS and network cards) as well as the storage arrays. Today there are NAS heads (the server portion only) that work with external storage. You guessed it, one supported configuration is a NAS head attached to a SAN that hosts all the storage devices.

So, we "no bid" on the opportunity to provide rules-of-thumb to make your choice appear easy. Rather we are anxious to provide the detailed, individual analysis that these important decisions demand.

The future of SAN

The chart shows a simplified map of the SCSI-3 architecture (click to enlarge in a separate window).

The feature we want to focus on is the choice of implementations in the physical network transports. The old SCSI Parallel Interface is still supported over on the left. As we mentioned earlier, once the architecture separated out these layers, all sorts of other possibilities arose and SANs were the result.

Initially, almost all SANs used the Fibre Channel (FC) protocol (a triumph in misleading naming, since it can be carried on optical fiber or copper wire). FC was preferred because of its speed; rated at 1Gbps it is an efficient transport for which the rated speeds don't exaggerate real throughput as much as most others do. If more speed is needed several FC channels can be combined (or trunked). Since the one thing you need for a SAN is speed, FC was chosen despite the high cost of the Host Adapters and other equipment. As a result, early SANs were only to be found in large, complex settings where the high cost could be justified.

At the right end of the diagram are two newer protocols that have something in common. The RDMA protocol using InfiniBand and iSCSI using IP are both capable of carrying storage traffic and regular network traffic over a single, common network. This is a pointer to the future. In the "old" days, the storage network carried data between servers and storage devices at speeds that were considered exotic and near the limits of network technology. Meanwhile, the network between the application and the client (end user) carried text or HTML at relative speeds that, compared to storage traffic, invoke images of hares and tortoises. No one in their right mind thought about mixing these data streams in the data center networks (although a thoughtful person might have reflected on the fact that in the WAN the common carriers mix them all the time.) But the "old" days are over. Already we are in a world where a single end user viewing a full-screen video is consuming the sort of bandwidth we used to reserve for storage networks. And the requirement of such a video stream in terms of guaranteed delivery and frame sequence may be more challenging than any server to storage-device exchange. In this new world, expect a strong trend towards the type of network convergence enabled by InfiniBand and iSCSI/IP.

However, it is iSCSI that promises to be the future star of the SAN world, if not the whole networked storage world. Here are some of the things it has going for it:

• It uses IP over Ethernet. Therefore, it can use the low-cost NICs, switches and cabling that have become commodities due to their ubiquity. The high-cost reputation of SANs, created by early Fibre Channel implementations, could be turned on its head.

• Ethernet is fast enough and getting faster. 1Gbps Ethernet is now commonplace. It doesn't rival 1Gbps FC due to differences in transport efficiency. However, if 1Gb Fibre Channel was fast enough for the stratospheric SANs, it is easy to see that 1Gb Ethernet is plenty good enough for the needs of many down-to-earth users. Of course, Fibre Channel is doubling its speed -but Ethernet is threatening ten-fold increases in bit rates to steal the crowns for both real and rated speed.

• IP/Ethernet are familiar. To install Fibre Channel SANs, organizations had a lot of learning to do; from new terminologies to new protocols to new physical installation techniques (like learning not to look into an optical adapter if you value your eyesight.) Since most organizations don't have a surfeit of IT people waiting to learn new skills, this provided another barrier and increased the risk of choosing the SAN direction. Cancel these concerns for an iSCSI IP/Ethernet-based SAN.

• SANs could become appliances. One of NAS's keys to success was the appliance mentality. But exotic users don't buy appliances and so early SAN products were directed at the specialized upper-end of the market. iSCSI could democratize SANs to the point where the market is big enough for vendors to take a different approach. Maybe SANs won't be single box appliances like some NAS solutions. But, compared to early SANs they will come very close.

Conclusion

Let's try to be as net as possible:

• Storage networks are the only way to go.

• Both NAS and SAN provide attractive, mature solutions that deliver the essential benefits of storage networking.

• Choosing a good solution today is easy - because there are so many good solutions. Choosing the best solution today is difficult - because there are so many good solutions!

• In evaluating different solutions, administration and management costs should be dominant (see the next two articles).

• iSCSI/IP SANs could have a fundamental effect on your future choices.

If you have any doubts about your storage direction, click the buttons below. On the other hand, if you are very confident about your storage network strategy - why not click the buttons anyway? In an area as important as this there's nothing wrong with a second opinion - especially when it is free.

... on how Datatrend can assist with your storage challenges

Administering and managing storage

IBM's TotalStorage Productivity Center illustrates what's possible

In this article we are going to look at a specific product. The objective is not to sell you that specific product (although it is an excellent solution for many needs). Rather we will use it to illustrate directions in today's products that are moving towards the synthesis of the next generation of storage management (ILM covered in the next article).

IBM's TotalStorage Productivity Center (TPC) provides excellent illustrations of many of the trends in storage management and the best practices that they support.  The first clues can be found in the name of the product.

Trend 1: Integration through architecture and standards

The composite word "TotalStorage" represents a major effort by IBM to evolve a very broad range of hardware and software into an integrated solution that is directed at your overall goal of lower TCO through better total storage management. Since the previous sentence sounds suspiciously like marketing fluff, let's substantiate it with more detailed illustrations.

"Total" means comprehensive. Real-world storage management has become complicated. Much experience has taught us that the place to go for a long-term solution to complexity is architecture. A clear, logical picture of what we are trying to build is vital. The chart on the right shows the one of the architecture diagrams that is being used to drive the design and development of TotalStorage. TPC fits in the Storage Infrastructure Management module. One of the benefits of a good architecture is the clarity it brings to development. The developers of TPC have a clear knowledge of their role and the interfaces to the other modules; confident in this knowledge they can work at full speed to do their part of the "total" job.

"Total" means all vendors. The days of IBM proprietary solutions are long gone. TPC is designed to handle all your storage devices from all vendors. This is achieved by adhering to industry standards. For example, the TPC agents that collect and monitor data about devices conform to the Common Information Model (CIM). Thus, TPC can collect data from any CIM-compliant device from any vendor.

"Total" means all aspects. TPC consists of four different parts, each of which manages one aspect of the infrastructure:

• TPC for Data (formerly Tivoli Storage Resource Manager)

• TPC for Disk (formerly the Performance Manager feature)

• TPC for Fabric (formerly Tivoli SAN Manager)

• TPC for Replication (formerly the Replication Manager feature)

Between them, these modules cover all aspects of storage infrastructure management. Seeing the name changes, the cynical reader may suspect "badge engineering". In fact, these name changes are the visible part of reorganizing the IBM and Tivoli software within the new architecture. As such, they change not only the badges but some crucial internal plumbing. For example, all functions in the total architecture need to collect data and use software agents to do so. If each function implemented its own agent your storage devices would spend more time executing agents than handling data! On the other hand, a rigid scheme that required specific hyper-agents to be installed on all devices before any function could operate would be inflexible. The actual solution is much cleverer and provides much more flexibility. Common Endpoint services allows each collaboration between functions and modules to manage the agents and optimize the way data is harvested across the storage network. Whatever combination of TotalStorage modules you have installed and whatever sequence you install them in, Common Endpoint services discovers and uses a single Endpoint Manager function to manage this optimization. As you can see, this is a direct result of the architectural changes and, in fact, TPC was the first function to implement this advanced part of the architecture.

In summary, TPC demonstrates our first trend - a strong trend towards integration across all functions and many product vendors. It achieves this with architecture and standards.

Trend 2: People productivity

TPC is, of course, concerned with the productivity of your storage assets. However, our priority here is the productivity of the people who administer and manage those assets. This priority is not difficult to understand if you reflect on the statement in the overview article that some people believe that every $1 spent on hardware incurs $8 in lifetime management costs!

The story of the IT operations and planning productivity has often been like the story of the cobbler's shoeless children. After all, if someone wants to increase the productivity of warehouse personnel, they call in IT to implement warehouse automation. Then the same IT specialists go back "home" and create an inventory of their storage devices manually! Here are some illustrations of how TPC is the warehouse automation for your storage assets and then some.

Discovery: TPC's agents crawl over your storage assets and find out what is there. From this elemental data a structured database of assets is created. Total manual effort - zero.

Reporting: TPC provides hundreds of standard reports on asset inventory, utilization, and trends. You can get organized reports on what you've got, where it is and how much it is used. You can also get graphical summaries of your network topology. Total manual effort - zero.

Monitoring and exception reporting: TPC notifies the administrator if something needs immediate attention - for example, if free space on an array drops below a certain threshold. It will send an email and/or call a pager as well as reporting the event in logs and to the system console. Total manual effort - zero.

This automation for IT administrators matches the best applications that IT provides to the end-users. In other words, the cobbler's children now have shoes!

Trend 3: Policy-based

How can TPC know what level of free storage constitutes grounds for an alert? It provides the structure for an organization to establish a set of policies, quotas and constraints. It can then constantly monitor operations against these policies to report any violations. It can even project trends to warn of potential future violations.

Trend 4: Autonomic action

All the prerequisites are covered in the previous paragraphs; TPC is constantly monitoring the status of your storage subsystems; through your policies it know when something needs action. All we need to do is close the loop and have the system take that action and it will become self-managing.

The actions themselves are generally outside the role of TPC. That's where the overall architecture of TotalStorage comes in. For example, it is the job of Tivoli Storage Manager (TSM) to allocate new extents, to move data to different tiers in a hierarchy of devices or to delete data. TSM will do all these things under manual control. But it can also take its instructions from TPC. So, based on the policies you set up, TPC can detect if free space is low and alert TSM. Using more of your policies, TSM can act on the alert by allocating space, moving data to tape, erasing old data or a host of other possibilities.

Conclusion

Storage management is very important. It requires software that gives a high priority to both taking a total approach and to increasing the productivity of the people involved. With some assistance from blue highlighting, the very name of our case-study product - TotalStorage Productivity Center - encapsulates the need. Careful study of this product shows a state-of-the-art implementation of the key themes:

• The need for a powerful, comprehensive architecture to enable integration and rapid evolution.

• Heavy emphasis on people productivity.

• Providing facilities that allow the user to express a rich set of policies.

• Abilities to:

  • Automatically monitor a complex environment.

  • Automatically discover changes to the environment.

  • Compare current conditions to the database of policies and autonomically take action to enforce the policies.

In fact, these here-and-now capabilities are the perfect introduction to the future as described in the next article on Information Lifecycle management.

Click the InstantDEMO symbol to see a streaming video of a demo related to this article. (In this case, the product being shown is not TPC but is related to storage management.)

... on how TPC could fit into your plans

Information Lifecycle Management

What's all the fuss about?

Let's start with a definition:

ILM is a way of managing information that meets the needs of the organization while minimizing the cost. It is superior to previous attempts because it understands how the value of information changes throughout its lifecycle and uses that understanding to manage the underlying data in a more cost-effective way.

This definition displays two characteristics:

• It is rather generalized - and, therefore, somewhat content-free

• The definition personifies ILM. It talks about ILM "managing" information or data as if it had a life of its own.

These characteristics are there on purpose, because they will allow us to investigate what the real, long-term significance of ILM might be. Let's start by attacking ILM based on this definition.

"I've heard it all before"

It is not unusual for "new" concepts in IT (not to mention the rest of civilization) to be met with cynicism. Sometimes the cynics are right and history proves the new idea to have been mere hot air; in the other cases, the cynics turn into supporters and claim their previous remarks were misunderstood. There are plenty of budding ILM cynics today; however, the author advises caution before joining their ranks! Here are some common reactions to ILM:

• "It sounds like HSM to me!" Hierarchical Storage Management (HSM) was talked about as a concept forty years ago and even introduced as a product (IBM's DFSMS - HSM) over thirty years ago. ILM sounds like HSM because it is! At least - up to a point. But it is also more than HSM. Some people try to simplify the difference by saying that HSM classified data merely by age whereas ILM classifies information by business value. But those of us who were around HSM know that this oversimplification severely understates the sophistication and elegance of HSM even decades ago. A more effective answer to the cynic is "You're right! Part of it does sound like HSM - and that is a very good thing."

• "Not the old information versus data thing again?" The first discussions on data versus information probably took place several thousands of years ago; certainly, this was a hot topic in IT DB seminars in the 60s and 70s. Here we tell the cynic "Yes, that old thing again. Because the concept is just as important as it has ever been and there are many aspects that IT still struggles to implement."

So, as you think about ILM (a subject that demands deep thinking) don't be diverted by the familiarity and age of some of the concepts and components. After all, "light" is a familiar, old idea; but watching a laser beam cutting through a block of steel it's inappropriate to say "Not that old light thing again?". One of the strengths of ILM is that some of the core concepts and techniques are familiar and old. Just like the laser, it's how we use them that could produce startling transformations.

What is ILM?

There is no agreed definition. To some it is a broad, long-term concept. To others it is a narrow, existing product solution. To most it is something in between. We vote for thinking of ILM as the broad, long-term, uniting concept; that's what we assume in this article and that's why it could be so important.

ILM is more of a direction than a defined destination. Implementing it is more of a continuous journey than a set of events. Having it in one's organization is more about process and attitude than it is about owning specific products. However, this should be no surprise; these statements are usually true about the more important ideas.

In case the previous paragraph brings out the cynic in you, let's leave the big statements behind and identify the elements of ILM that make up the direction:

1. Value of information: ILM requires that value be assigned to each "piece" of information. The value of "emails" is meaningless. One email is a recipe for pound cake and the next one is about a $10M liability suit. The "average" value of these emails is no basis for managing storage. ILM, therefore, requires processes for assigning real-world value at a low level. While systems can help to classify information, human intelligence must be the final arbiter.

2. The changing value of information: This is where the "lifecycle" part comes in. A day-old invoice that is still outstanding needs different treatment from a four-year-old invoice that has been paid. Since the law states that I only need to keep invoices for five years, it is easy to say that a six-year-old invoice has no value and can be destroyed; but maybe that invoice still contains information of value to an analyst in the finance department. Having processes that can recognize the changing value of information is central to ILM. Once again, systems can help but business savvy is needed to truly understand value.

3. Servicing the needs of the organization: Saving money on data storage is easy. On the other hand, reducing total cost of ownership while meeting a complex web of service-level agreements for each type of information is difficult enough to make your brain hurt. In the past we have often used the sledgehammer approach; in order to protect and provide rapid access to the litigation email we have provided similar service levels for all emails - including the pound-cake recipe.

4. Breaking the connections that create complexity: Consider a single piece of data - like a line item on an invoice. This represents completely different information to several people; the billing clerk; the financial analyst; the database administrator; the manager planning storage capacity; the replication administrator. In the bad old days we tried to manage all these points of view by pre-planning and design. This was difficult enough when we only had a few "applications" with fairly structured data. Today it is impossible; we need to have each person able to go about their job independent of the others - but without tripping over each other.

If the previous list is what ILM needs to do, the following list points the way as to how to do it.

1. Taxonomy: If we are going to classify data and its value, we need a valid and powerful scheme of classification. In HSM data was classified mostly by age; this is clearly inadequate. In some early attempts at ILM you will find examples of classification that combines age with some other factors (like complying with retention regulations); these schemes represent progress but are also clearly inadequate to fulfill the broad promise of ILM. When Linnaeus invented his taxonomy for classifying plants and animals he could never have guessed how much influence it could have on scientific thinking and continues to have over 250 years later. In the world of classifying data and its changing value we are still waiting for our IT Linnaeus to give us the breakthrough ideas that seem so obvious once we see them. But now that ILM has exposed the classification vacuum it is only a matter of time before it will be filled. While these data-value classification schemes may not last 250 years, for many years we may wonder how we ever managed without them.

2. Service-level agreements: The discipline of SLAs has become much more common in specifying how IT should operate. As part of implementing ILM we will need to spread the SLA mentality much deeper and wider. How quickly should this data object be available? How quickly must we be able to recover it when the main copy fails? How quickly should we be able to access it when it is five years old? We need clear answers to these questions for the invoice, for the litigation email and for the pound-cake recipe - as well as for a huge number of other information objects. Only when these service levels are validly and clearly expressed can IT set out to provide the lowest cost way of meeting them.

3. Architecture: When an earlier article looked at the evolution of SCSI we remarked that it was the architectural changes in SCSI-3 that unleashed the full potential and enabled the SAN revolution. That's how powerfully architecture can work; we need it to work with equal power in ILM. However, in ILM we aren't looking for a system architecture; we need an architecture to clarify the logical roles of management, users, IT and other players in, for example, classifying and assigning value to information. This is a process architecture. We are not starting with a blank sheet; much progress has been made in areas like database administration, replication and recovery where we have seen new roles emerge and a clearer definition of those roles. As the ILM journey continues we can look forward to a synthesis analogous to SCSI-3 that will unleash new possibilities. Architecture defines logical roles, thus making it possible to separately develop different physical implementations of each role. In our SCSI-3 analogy, the logical definition of the transport layers allowed Fibre Channel and iSCSI implementations to develop in parallel - even in competition. As the architecture of ILM matures and roles become clear, executing those roles will become simpler - even though the environment we are executing in will become massively more complex. The architecture will allow each "layer" to get on with its work without worrying about how the rest is done. This is how ILM will break the connections that today make information management appear hopelessly complex. This is how, for example, a replication or recovery administrator will be able to optimize their role without having to have a hundred meetings with the people responsible for assigning value to the information.

4. A multi-dimensional network of policies: Earlier we remarked that, while systems can help to set policies, it is the human judgment and expertise that will make the difference between adequate and excellent. But that insight will come from many people with different points of view. Take data retention as an example. Many people will have valid input on how long their information (and, therefore, the underlying data) should be retained. Operating department managers, human resources, financial planners and the legal department may all have a valid view of the need to retain the same piece of data; these views could be very varied. The practical way to capture and use this input is as policies. Once again, we have already seen a lot of progress in this area (see previous article on TPC). As the ILM journey continues the major contribution of the hardware/software components will be to assist in the recording, organizing and combining of what will become a much richer database of overlapping policies. The time-based HSM policies of the 80s look crude compared to the most advanced ILM policies of today; but these will, in turn look unbearably naive compared to the policy-based systems to come.

So what?

That's always a good question on which to finish! Especially when the topic is so broad and futuristic. So, let's try to answer some variations of it directly.

So, why should I care? There's a big challenge to meet. Storage demands are going up geometrically. Total storage costs are going up steadily. Current practices are clearly rather crude for today's challenges and simply inadequate for what is to come. ILM is the only direction that meets the challenge head on.

So, what can I do about it? Depends a lot on your role. But here are some generally applicable ideas:

• Everyone in general management and/or IT should invest a little time in following the ILM story and thinking about how it affects their organization. Rather like sparing some time to think about the Internet in the 90s - in the 00s ILM should be constantly on your mind.

• Learn from compliance. If your organization has been affected by regulations (like SarbOx or HIIPA) think about how much effort it involved; how different would it have been if you could have just adjusted some retention rules in your ILM database of policies? This can be a telling, real-world case study.

So, what should I avoid doing? Don't try to "buy" ILM. ILM is a direction, a mentality, a set of procedures. These don't come as a product. But as you do buy products, evaluate them against how much they take you down the ILM road. It is no coincidence that we chose to feature TPC in the previous article. It is an example of a real-world product you can buy today that takes you a few steps on the journey to ILM; a product that has a well-architected, well defined role that fits into a bigger architecture; a policy-based product that communicates with other policy-based products so that, in total you are investing in building your information policy database.

Conclusion

Forecasting is a hazardous pursuit - especially when it involves the future.

Despite this truism, in this article we have stuck our necks out and predicted two things:

• That the storage of data and the management of information is going to be a major battleground, if not the major battleground in IT over the next few years. This is where an IT competitive edge will be won or lost.

• ILM is going to be of fundamental importance to winning that battle. Those that lead in ILM will lead in IT TCO, ROI and competitive edge.

That's our big predictions. Do you agree?

... how you could get started on the ILM journey

TechTip: Improve your SANs with zones

What are zones? how do they work? Why should you use them?

By Rob Myers, Senior Systems Engineer, Datatrend Technologies

Storage Area Networks (SANs) that may have begun with a clean and simple installation tend to grow in size and complexity as their usefulness becomes more understood. With that growth comes a decrease in their manageability. Zones or “zoning” is the way to counteract this tendency.

A zone contains at least two zone members, such as servers or storage devices, which are permitted to communicate with one another. Because of the zone, the zone members can only communicate with other zone members. If a zone member is in multiple zones, it can communicate with any of the members in each zone.

A zone may either be a “soft” or “hard” zone. A “hard” zone is composed of a set of fixed port designators (actually the WWNs, or world-wide names, for those ports) on the SAN switch. Whatever is plugged into one of those ports becomes a zone member. “Soft” zones (or name server zones) are composed of a list of WWNs which are maintained in a name server database that is updated each time a server or device joins or leaves the SAN fabric. These WWNs come from the adapter interface at the other end of the fiber cable. By construction and convention, WWNs are to be unique.

The benefit of “soft” or WWN zoning is that a device can be moved to any other switch port in the SAN fabric, and zoning will continue as designed. However, if the fiber adapter or HBA (host bus adapter) has to be replaced, or the fiber cable has to be changed to a different adapter, this changes the WWN and that member will no longer be in the zone. The benefit of “hard” or port zoning is that devices can easily be located on a switch or director, and failures can be more easily detected and remedied.

Other Benefits of Zoning

• Increases Security Zoning provides higher security measures by enabling an administrator to separate servers and devices into zones, separate zones into zone sets, and activate or deactivate zone sets as required to restrict communication between devices. Restricting communication ensures that accidental data transfer or interference between devices will not occur.

• Separates Operating Systems Zoning provides barriers between systems with different operating system environments or uses. For example, it is often necessary to separate servers and enterprise storage that use different operating systems because accidental transfer of information from one to another can delete critical data. Zoning can prevent this data loss by grouping devices with similar operating systems into separate zones.
• Improves Administration and Maintenance Zones separate groups of ports within the fabric into smaller, more meaningful sets that can be identified by the function performed or the data contained and processed within it. Zones can also be set up to perform maintenance or testing on specific devices without interrupting devices attached to ports in other zones. Zoning improves the accessibility and reliability of the data by making it easier for administrators to manage the attached devices as identifiable groups. Zoning becomes even more beneficial in a multiswitch fabric to control device fan-out and maintain performance.

Planning for Zones

Determine if zoning is right for your fabric. If so, evaluate if the purpose of zoning is to differentiate between operating systems, data sets, user groups, devices, processes, or some combination thereof. Plan the use of zone members, zones, and zone sets accordingly. Some storage devices implement LUN masking. If these are present, the LUN masks must complement and work with the zones to be implemented.

Planning and implementing the zoning feature can be a complex and difficult task, especially for multiswitch fabrics. It is recommended you obtain planning assistance from an technology services organization which specializes in SAN/storage before implementing a director or switch zoning feature.

Contact us | Visit Datatrend website

All trademarks, registered trademarks and service marks are the property of their respective owners.
IBM, the IBM logo and other referenced IBM products and services are trademarks or registered trademarks of the International Business Machines Corporation in the United States, other countries, or both. All rights reserved.

UNIX is a registered trademark of The Open Group.
Windows is property of Microsoft Corporation.

Brought to you by Datatrend Technologies Inc.

6815 Meadowridge Court, Alpharetta, GA 30005