Three Messy Realities of Cloud Expansions and how Big Data helps

by Lilac Schoenbeck

As cloud becomes increasingly mainstream, companies are moving long lists of existing and new applications into external clouds. At the same time, shadow IT is emerging from the darkness as formerly “rogue” teams recognize corporate IT can, in fact, deliver agility while addressing security and compliance requirements. To optimize efficiency as well as cost, all of these workloads are being consolidated under a small handful of cloud contracts, often with cost-saving resource reservations and central management.

In tandem with growing numbers of cloud workloads, the teams of administrators managing these systems are growing. What was once handled by one or two people, now must be managed by a team, with members located down the hall – or around the globe. Ultimately, companies are finding that the cumulative growth of workloads and administration teams results in increasingly complex cloud management.

The good news is: Big Data can help. When Big Data is discussed in the context of cloud, it’s often to often analyze whether Big Data belongs IN the cloud. That is, can and should cloud infrastructure be used to help with Big Data storage, processing and analytics. There are very few conversations about how Big Data can be used FOR cloud management. How should service providers leverage the billions of data points living within their cloud infrastructures to help customers trying to manage their usage? Are the specific attributes customers should look for in a service to avoid common pitfalls?

There are three messy realities of expanding cloud footprints that Big Data can tidy:

Reality 1: Chaotic directories and naming conventions

In some ways, cloud is like a new laptop. Most users of a new laptop will promise themselves that they will maintain an idyllic directory structure and consistent naming conventions. They intend to keep things tidy so that we can quickly find what we need, when we need it. Despite their grandiose plans, almost immediately the laptop is completely cluttered because human nature cannot be denied and most people haven’t the time, memory or patience to stay organized.

The same thing happens in clouds. Teams set forth with orderly directories and naming conventions, but those teams grow, staff turns over, and new opinions develop on ideal protocols – and the structure begins to crumble. A shift from version numbers to dates, from Simpsons to Big Bang Theory server names, or from one snapshot policy to another can quickly result in a veritable mess. Eventually, members find themselves wasting significant resources trying to gather the workload information they need to execute tasks, or keep straight their progress updating operating systems.

Rather than trying to change human behavior, cloud service providers must make it easy to locate items within the environment, whether by virtual machine (VM) name, change activity, performance details, firewall rules or other something else entirely. Cloud customers should seek platforms that leverage Big Data to deliver the same types of search tools available on consumer technology platforms. Look for instantaneous full-text searching capabilities across the entire cloud environment. If that is coupled with alerting capabilities that alleviate the burden of monitoring, it is all the better.

Reality 2: Complicated resource optimization due to mysterious historical data

No one wants to pay for cloud resources they are not using. Nor do they want to end up with an outage because of inadequate resource allocation. When optimizing resources and forecasting costs to “right size” their virtual machines, administrators frequently reference historical performance charts and call upon their own memories to determine whether a spike or drop was associated with a resource change or application behavior. These determinations are particularly challenging when new team members are assigned the task, as they have no historical context. No doubt, the accuracy of this method is less than ideal.

With the analytical power of Big Data, cloud vendors can provide teams with correlative insights that map event histories onto performance graphs. Cloud management tools can present “stock split”-like markers at critical points that remove the guess work of retrospective thinking. This not only saves administrators time and minimizes erroneous conclusions, but also enables varying users to accurately resize workloads, regardless of their tenure and direct knowledge of the VM.

Reality 3: Conflicts between teams

As teams grow, aligning the timing of workload maintenance between different individuals becomes challenging. Actions need to be completed quickly, and administrators must be confident their changes will not impact others in the organizations. For that reason, it is critical to have an aggregate view of all changes to an environment. Again, Big Data can help.

Cloud administrators often use management portals, as well as a set of additional tools, to roll out their updates, monitor workloads, and make resource changes. To ease the burden of communication between fast-moving individuals on geographically-disparate teams, service providers can consolidate these activities into a single view. With that aggregate view, administrators can double-check on the status of an update, see who is responsible to the change, and drill into the details without resorting to email and chat confirmations every time.

Scalability is one of the biggest promises of cloud, but teams are quickly experiencing growing pains associated with these messy realities. The truth is, very few cloud service providers commit resources to designing and developing native management tools that address complexity issues. In fact, in a global survey conducted by Enterprise Management Associates more than half of cloud customers reported they require better management dashboards (Infographic). Many vendors promise capabilities, but then leave customers to navigate a vast “ecosystem” of applications and third parties with the assurance they will find a solution that works for them. The result is wasted time and unexpected fees.

Whether getting started in cloud or planning an expansion, companies should carefully evaluate the tools cloud vendors provide. They should be given a comprehensive demonstration of what is provided by the vendor, and have a good understanding of where add-on software means additional costs or configuration.

Customers should ask vendors how they help address growth challenges, specifically. Do they provide an intuitive dashboard teams can use to manage resources across expanding data center locations? Does that dashboard expose detailed historical and real-time data on usage, performance, costs? And how does it call upon, correlate and present that data? Is instantaneous full-text search included?

By understanding common challenges, teams can avoid pitfalls of cloud expansion. The key is choosing a cloud provider that will help anticipate those challenges, and also deliver the technology needed to overcome them.

Lilac Schoenbeck is the vice president of product management and marketing at iland.


Leave a Reply

WWPI – Covering the best in IT since 1980