December 4, 2013
A packed room of excited enterprise IT pros attended our talk at this year’s AWS re:Invent,“The Science of Choosing EC2 Reserved Instances.” In the talk, our Vice President of Product Development, Toban Zolman, walked through how Reserved Instances are used, how to make the best possible Reserved Instance purchasing decisions, and how to significantly reduce the time needed to make those decisions. Check out the talk below:
November 26, 2013
In a talk earlier this month at Defrag Conference 2013, Cloudability CEO Mat Ellis discussed the changing role of the CIO in the wake of the cloud revolution, and how a potentially challenging transition between technologies can actually represent a golden opportunity: “We think the CIO role is poised to take on a lot more power,” he said, “But only if CIOs recognize that their role now is to drive integration, not information.” Check out the transcript of his talk below:
I’m Mat Ellis, CEO & Founder of Cloudability. We’re based in cloudy Portland, Oregon. Eric asked me to take a few minutes to talk to you about a big shift we’re seeing in the role of the CIO.
The growth of the Internet of “Things” gives us some interesting data about how the CIO’s role is changing. And, for many companies, this is a change they want — and need — to encourage.
This Internet of “Things” is making us look at the world— the technology world at least— very differently. We are managing more and more devices that feel and look nothing like a computer or even a phone. Even the servers we use to run our businesses are increasingly anonymous, un-named, disposable, and virtual.
In fact, today, technology infrastructure is beginning to resemble a neural network. A network pummeled with bits of data coming at it from millions of devices — these “things” —that few of us ever imagined would be computerized or connected.
It’s this evolving interconnectedness that is forcing us to radically change how we manage the flow of data between all the things: objects as disparate as cars and traffic lights, lamps and power grids, paint chips and bridges, and so on.
We are going to have to rethink how conventional rules are applied in a world where so many things need to talk to each other without humans getting in the way. These challenges of interconnectedness are playing out in parallel inside the enterprise.
CIOs have an once-in-a-lifetime opportunity to provide the leadership needed to guide their companies through these challenges; done right, the “I” in CIO becomes more about managing integration and less about managing information.
In one of his blog posts, Eric Norlin has called out the three defining characteristics of this network. He coined it “a bigger network of smaller things.” They will be:
(1) decentralized like the Internet,
(2)interconnected nodes in a dispersed central ecosystem, and
(3) fractal, where the structure looks similar as we zoom in or out.
We can already see what’s driving this: a wonderful mix of cloud computing, data and APIs that glue it all together.
But inside the enterprise, this is pure chaos. Rogue apps, devices and clouds are causing more and more “islands of information” to form. And while startups may thrive on chaos,enterprises struggle with it.
A natural response is to deal with the chaos by blocking anything that isn’t already part of their legacy system. Those of you who have been through the BYOD battle will have had a taste of this already.
Those enterprise who have got past this reflexive response are struggling with how to manage all of this new, distributed stuff, and how to integrate it with the existing stuff. And it’s this chaotic struggle that is holding back innovation and growth in enterprises.
Former Forrester Research president and research director Dave West may have said it best in an interview with Joe Stangarone on the MRC blog:
“Application integration problems are a top reason why businesses — and their enterprise architects and project managers — can’t deliver business innovation at the speed demanded by customers using all these application platforms.”
On one hand, you have IT whose natural inclination is to make the infrastructure as consistent, cost effective and solid as possible… which often translates into descriptions such slow, rigid and bureaucratic. On the other, you have divisions and employees — the nodes, as Eric would call them — who can seem to have tunnel vision at times, focused on only one thing: moving faster.
The CIO’s opportunity sits smack dab at the intersection of what’s best for the overall network and what’s best for the nodes.
I read an article recently that quoted Mark Thiele on this topic. In the article, Mark pointed out that integration is not only a technological opportunity, it’s also an organizational and business one. In the past, the roles of IT and business were separate. Two parts of the same network operating as islands.
At Cloudability, we get to see how the sausage making process of enterprise cloud adoption really works. We see a lot of political discussions on both sides — pro- and anti-cloud — about the wrong things. The people who spent their careers dedicated to building out world class data centers full of hardware that just a few years ago was everything the company needed, are finding it hard to embrace it when business units increasingly bypass them, ignore them, or unfairly compare internal services to external ones.
To go for Cloud services is often more expensive in cash terms right now, but the reality is, for that premium, you are getting less risk and more speed. So both sides need to focus on determining the right thing to do— not every project should go on the cloud, and not every project should go on the public cloud.
And the discussion should be, what are the numbers driving those choices? Not “let’s get rid of the Cloud” or “let’s get rid of internal IT.”
We’re already seeing greater integration of IT into business units as those on both sides begin to better understand each other’s roles and how they impact the overall growth of the organization. In fact, a recent IDC study showed 85 percent of respondents said IT has become a more valuable partner compared to three years ago. A bigger network of smaller things. And it’s that integration — of IT into business— that explains why the Chief Information Officer needs to become the Chief Integration Officer.
To the CIO, cloud computing and APIs are code words that describe a more fundamental shift from servers to services, from Capex to Opex. We feel it’s very similar to the move from mainframes to microcomputers. And it’s driven by the same forces: businesses want more speed, and more flexibility.
Today, you pay for everything up front. You own the hardware and the software, as well as the long term co-location, support and network contracts. And you employ staff to manage and maintain these systems. Well, not for much longer. You will end up with all of the above presented to you as a service: everything, from individual servers to whole applications will be provided as a service.
This has happened before. Over the last 30 years manufacturing industries have evolved from horizontally integrated supply chains – where you owned the factories, the source of raw materials, and even sometimes the cows that made the leather or the trees that made the rubber – to a vertically integrated supply chain based on a global network of suppliers who can make anything from rivets to large, complex components for you.
For example, look at how Toyota builds their cars. They buy components such as car seats from many different manufacturers. Orders are placed every day to a mix of vendors, based on demand. Each seat is identical except for the price, which varies according to the commercial terms in place: minimum order sizes, lead-times, annual commitments, etc.
It takes a lot of serious technology and people to stay on top of that and win.
Another example of this is the most recent US presidential election. It’s been widely reported that a critical advantage Obama had over Romney was technology. If we look at each implementation, they are wildly different.
Full disclosure: The Obama for America campaign was a customer of ours, so we have to be careful what we say here.
What is public is that the OFA team took a completely cloud based approach to the problem. This allowed them to spend almost all of their time focused on algorithms and data. And they were able to scale their successes very quickly when they had them.
The Romney campaign on the other hand took the same approach as every previous presidential campaign had taken. Their project ORCA needed a lot of effort just to get the infrastructure in place, before any code could be written or data crunched. Unfortunately, all this effort failed to pay off: on election day it quickly became overloaded and was unavailable for long stretches of time, just when it was needed most.
It’s generally accepted that this made a really big difference in Obama’s campaign. They had access to the data and could put people in the right places, whereas the Romney campaign didn’t and couldn’t.
I think this is the best and most public example to date about what will happen to you if you don’t get your ducks lined up on this integration.
With the proliferation of cloud applications and personal devices into the enterprise — not to mention existing data silos and platforms already inside the enterprise — organizations are generating more data than ever before. Data that must be corralled and managed. Data that must be tied directly into the company’s revenue and growth.
CIOs will need tools and systems that allow them to visualize and guide the growth of this neural network of things. Think of it as a compute supply chain, with the CIO managing the vertical integration of this chain. Once they can see all of the nodes, they can operate a more dynamic, flexible compute supply chain.
Because of this, we predict the CIO’s office is poised to become smaller but a lot more powerful. This will only happen if CIOs recognize that their role now is to drive integration, and no longer to provide the infrastructure for corralling information. Once they realize that, a world of new opportunity opens up for them and the people they serve, as well as their organization overall.
It’s a transformation whose time has come.
Viva la Chief Integration Officer!
November 19, 2013
I was looking through the list of workers connected to our system one day when I noticed something a little troubling: an IP address that I didn’t recognize. We have an elastic infrastructure here, so it wasn’t too weird, but based on the number of workers connected and running on that box, it still didn’t feel right. Was someone listening in on us?
Dealing with Cloud elasticity
This is a pretty common problem for public cloud users. The elasticity of the public cloud allows you to have just the right number of servers for the load your app is taking— you don’t have to over-buy servers to anticipate more users. However, this ever-changing environment often presents difficulties for Ops teams. Often used to used to controlling the infrastructure, these teams have a hard time adjusting.
Our infrastructure at Cloudability is pretty elastic, too. In fact, applications can control themselves! Developers and apps can control their own scaling patterns, which lets us take full advantage of the Cloud.
So how do we deal with scenarios such as the aforementioned rogue IP, given that our environment is always changing?
Posted in Tech
November 8, 2013
In a world where lots of people in an organization can spin up AWS resources and there are multiple stakeholders working to manage fluid cloud-based budgets, Cloudability provides a common lexicon for cloud spending. To ensure that your AWS spending data is available to everyone at your company who needs it, make sure to give Cloudability access to the following folks.
Granting stakeholders direct access to AWS spending and usage data allows Operations to get out of the way of AWS-centric teams to whom agility is key. By providing a shared Cloudability account to all of the stakeholders in their organization, Operations can give these stakeholders the tools they need manage their costs independently. At a macro level, Operations can use Cloudability to profile their total AWS infrastructure, by filtering their costs by department in a Cloudability Cost Allocation Report. They can also use the Reserved Instance Planner to look across entire Consolidated Billing accounts to determine where there are opportunities for cost savings.
Access to AWS spending challenges engineers to improve their efficiency. Engineers can optimize when they see the impact that a design decision has on the cost of a product by creating a “cost by product” Cost Allocation Report. They can also look at the daily spend trends in the Cloudability emails to see how deployments affect costs. We hear regularly from our customers that providing insight into these costs promote engineer efficiency in product design.
Product and Business Owners
Access to AWS spending gives Business and Product owners the data to build and manage budgets. These types of people generally use Cloudability to get a weekly report of their progress against plan, and to build detailed monthly reports accordingly.
Cloudability gives Finance the ability to understand and control costs while minimizing the need for manual Excel work. Without requiring reporting work from Operations, they can allocate between business groups, see how groups track to their budget, and perform monthly/quarterly close much faster. They can build Cost Allocation Reports according to product, internal service, department, group, environment, region, and more, in order to track the sources of all AWS costs with accuracy and ease. Then, they can use Cloudability data as the starting point for conversations with the rest of the organization.
Posted in Tech
November 4, 2013
Nothing can save you money on your cloud costs like using Reserved Instances; they require no technical optimization effort, no dev ops wizardry, and no auto-scaling infrastructure. Just write a check, and you can save 60% — as long as you’re buying the right RIs at the right time.
Throughout our Reserved Instances series, you’ve learned all you need to know to purchase Reserved Instances confidently and competently. Now that you’re ready to launch into your cost-optimized future using RIs, take one final look at these four key tips for using RIs as effectively as possible:
One: Make frequent purchases
Making frequent reservation purchases is one of the easiest practices to help maximize cost efficiency. If you purchase reservations often, you will have a steady cycle of reservations expiring; this allows you to purchase new reservations appropriate to your current needs.
Two: Don’t over-purchase Heavy reservations
Instead of over-purchasing Heavy reservations, utilize Light and Medium reservations, which are better suited to handle volatility. The fact that Heavy reservations are pre-billed at the beginning of the month means that you pay for the entire reservation whether you use it or not. By charging hourly, Light and Medium reservations spare you this potential loss on volatile projects.
Three: Utilize Light reservations to hold capacity in specific Availability Zones
If capacity reservations are important to you, you can utilize Light reservations to hold capacity in specific Availability Zones. Light reservations work well for this purpose, because while they’re inexpensive, they still guarantee you capacity in case of an AWS outage.
Four: Calculate your needs correctly
One of the most common mistakes when purchasing AWS Reserved Instances is calculating your RI needs according to overall utilization rate over a given period of time. However, this is an inaccurate calculation method with the potential to misinform your RI purchases. The best RI purchasing decisions are based on hourly instance counts of each instance type per availability zone— calculations that Cloudability is happy to do for you automatically, with our Reserved Instance Planner tool.
It’s time for you to start saving money with RIs. Log in or start a free trial of Cloudability Pro to put your knowledge to work with our Reserved Instance Planner, and optimize your cloud costs today.