This is a report back from Cloud Expo Europe 2013 where keynote talks from IBM, eBay, EasyJet to Morgan Stanley gave a view of how far the cloud has been adopted. In short, the cloud has ‘landed’; it is no longer hype or on the bleeding edge but is being used in organisations both large and small. For some businesses it is still early days but many are carrying out implementations to understand the tools and processes they need in order to use the cloud, as well as the organisation changes required.
For the past two years Cloud Expo Europe has been held in London Olympia promoting the cloud from vendors of cloud solutions to those who are making use of the cloud. My first impression was that this year’s event was much busier than the one in 2012, with a lot more ‘suits’ attending the sessions rather than geeks.
Simon Wardley on IT Strategy
The main motivational kick-off session was impressive. Simon Wardley spoke about IT strategy, starting with a question to the audience asking what their IT strategy was; most people listening had a mixture of some real IT strategy together with “just following the rest of the industry”.
Simon’s view is that IT strategy should contain just ‘why’ and ‘where’ not ‘how’ or ‘when’. Like the army’s view of a battlefield strategy we should have a map showing us why were are fighting a particular battle and where we are heading in the grand scheme. But those in IT the business are usually reluctant to share the map they are following, and hence many IT strategies are flawed.
The example of Nokia was very useful: Nokia started as a paper mill, then rubber company before it went into mobile technology, and now where is it heading? Disruptive influences can alter the business strategy and the business if given time will adjust their strategy, but today many companies are unable to change quickly enough. Herbert Smith’s theory of componentisation and the “Red Queen” hypothesis from Prof Leigh Van Valen, leads us to the inevitable:
We have to change to stand still in a competitive market.
Simon reckons that a good IT strategy is based on the value chain and evolution graph of your business, providing you your strategic map. Co-evolution shows how we evolve best practices from the evolution of our strategy, and DevOps is an example of a best practice being created from the evolution of IT into the cloud.
Another useful view from Simon was that:
exponential change always catches us out.
For instance, it took Amazon took 4 years to get 3% of the global web hosting market, yet in 4 more years it will have 50% market share – an exponential growth which has taken many in the industry by surprise.
The disruptive technology adoption curve means the shift from product to utility is inevitable in all revolutions; look back at the mechanical and electrical revolutions, and the way in which products which were initially bespoke evolved into commodity or utility services over time. Cloud is the ‘utility’ of the computer revolution. By analogy, HP is today’s gas lantern street lighter, fighting against the introduction of electric street lamps. The cloud undermines HP’s traditional hardware and software product base in a way which means they may not be able to recover
Simon’s message: Use IT as a weapon against your competitors.
What is the Cloud? – views from five different organisations at Cloud Expo Europe 2013
What is the Cloud? according to Verizon
Verizon Terremark view the cloud as having the characteristics of independent architectures (i.e. your applications are separate from the physical hardware), with multi-tenancy, and with different controls (i.e. your ops teams need to perform different roles). Email and e-commerce are ideally suited to the cloud.
Workloads not currently in the cloud tend to be those that do not scale horizontally or are not fault tolerant. Many applications have not been designed to be fault tolerant or horizontally scalable; they rely on redundancy in the hardware rather than designing for anything failing at any time. Middleware, security concerns, dependencies between applications, management and performance make it hard to use the cloud for many applications.
- Physical security is a key concern i.e. with PCI, how do you protect against border laws? Who can access your servers? What happens to the data within different jurisdictions? In the future, advanced networking will be key to security, an example being software defined networks (SDN).
- Dependencies on the stack and infrastructure need to be removed before moving to the cloud.
- The management life cycle needs to evolve and change to remove the differences between your own data centre and the cloud. Future convergence between tools likely. Updates and patches are issues which have yet to be satifactorily resolved.
- Performance is critical for the storage and network. Enterprises still need predictable performance, and as a result there is still significant over-provisioning of hardware resources.
In the next 5 years it will be possible to host everything in the cloud through IaaS. PaaS will also extend to Database-as-a-Service or Identity-as-a-Service.
What is the Cloud? according to IBM
IBM believe technology is more important than ever and CEOs believe this is more important now than people or the market they are in. Big Data, mobile and ‘the cloud’ are the top 3 technologies the CEO is thinking about in 2013, according to the 2000 CEOs surveyed by IBM.
Security is a major concern in adoption of the cloud and mobile phones (the speaker referred to a 600% growth in security attacks from mobiles in the the year). IBM believe that open standards are key to allow intelligent workload automation and service orchestration in the enterprise cloud.
Resource orchestration (compute, storage, networks) is here; the next generation clouds will focus on how business orchestrations are created from business processes. For example, your car was originally built by one company; in 2013, car manufactures design and assemble cars with parts made by others. The cloud will be similar for enterprise applications but the standards are key.
IBM smart cloud services are now emerging. To sit on top of PureSystems (complete rack of hardware). IBM have built a SAP friendly public cloud using open standards. IBM believe their USP is in their implementation of the standards. They already provide 300+ cloud txns, to 1 million users with 13 billion security events per day. They provide 60 SaaS solutions and 5000+ private clouds. They believe in a very strong eco system as no one can provide all the business services.
Open standards are key to a heterogeneous multi-supplier enterprise cloud running cross-supplier business orchestrations.
What is the Cloud? according to Morgan Stanley
Morgan Stanley took us through their implementation plan for a hybrid cloud. They have 80,000 servers – a 10-fold increase in 10 years – and they need to get the cost benefits of using the public cloud but as a bank are hitting all the expected issues:
- Security of customer data and trading data
- Applications are not cloud ready: they are having to change them to scale out, use names spaces not fixed IPs, and be fault tolerant
- Centralized capacity management is a new challenge
- Lack of tools for managing cloud infrastructure (they are creating their own)
- Performance issues – not resolved but plan to use hybrid to help
- Data not all cloud ready
Currently they plan to have a hybrid cloud in use by next year.
Summary (Morgan Stanley)
Existing software applications do not fit well with cloud models, but can be evolved to fit.
What is the Cloud? according to EasyJet
EasyJet have started to make use of Azure to host new features, for example seat maps. In 2011 their marketing team decided it was time to give customers the choice of a seat as long as it did not delay boarding processes.
EasyJet started in 1996 with 2 planes and today it is the 3rd largest in Europe and 10th largest in the world. They only spend 0.8% of their revenue on IT so cost is a major issue for them. 0.8% is lower than any one else in any industry.
They have some off peaks in business for example one day in mid Jan every years sees a peak load that is 4 times the normal hourly load. Historically they have provisioned for that peak, but with Azure and new features being added to the cloud they have started to understand how to scale on demand.
Other challenges include:
- ‘Cloud Creep’ – teams adding new machines and storage with no control
- Cost control – how to ensure the cost is understood by the project / business unit
- New skills – staff need to understand how to work with the cloud, not Azure as such, but how to manage capacity
- Vendor management – no one to ‘beat up’ when things go wrong, as traditional SLAs tend not to exist with public cloud models (Microsoft have been very helpful, apparently)
- Scalability and Performance Management needs to be designed for
Azure has been providing 99.9% to 99.99% availability, but Azure can go down.
Performance testing and tuning is critical to keeping costs low. Revised capacity models are required to make it clear to every one what the cost of multiple CPU machines and large data stores is. For example, SQL Azure had a 150GB limit per instance.
Microsoft Azure has worked well for EasyJet, although performance testing and tuning are needed to keep costs in check.
What is the (private) Cloud? according to eBay
Today eBay can deploy a new application to 100 servers in 45 mins. They have 6000 servers, 2 billion page views per day, 23 million lines of code, 9 Petabytes of data.
There are 5 principles for a cloud (private or public) for eBay:
- Automate Everything
- Any application in any environment can be deployed anywhere
- Plan for failure
- Apply ‘proportional security’
Simplify your process – do not put complex manual processes or serial processes into the cloud, for example do not put your sign off process for change control into the enterprise pipeline.
Simplify your organisation – do not have silos of teams, get teams that have multiple disciplines. If you still need to have specialists get their roles removed from those directly operating the infrastructure.
Simplify your technology – go for a single SKU for your servers, use commodity hardware which will fail, consolidate your tools and access open source.
Automate, and if it cannot be automated (e.g. physical hardware installation) then go for solutions that are quick to add to your cloud; most vendors can now send you a rack full of kit. Remove any components that have no API. Remove anything that needs a manual reload or restart. Remove complex permissions or authentication mechanisms.
Things to avoid or do more of from an automation point of view:
- Service Calatogue – use restful APIs to drive the services not paper
- Ticket Driven Run Book – use automated process that audit what has happened but approvals and sign-offs are not required
- Complex Monitoring – use Hadoop/Splunk etc. to mine the data rather than systems that try and collect everything
- Central configuration management database – use distributed methods to identify the estate at any one time
- Financial Management of the IT, i.e. fixed allocations of resources through capex and opex – use pay as you go for the consumers
eBay have used OpenStack to build their cloud. They have avoided building clusters of resources dedicated just for QA or PreProd or Dev or Prod, but allow any environment to run on the same tin. This did require VLANs the span the entire network and then software driven control over the individual environments to create VLANs to isolate environments.
In order to plan for failures in the hardware, instead of having one single view of the hardware running the entire platform, eBay have split out functional components to map to specific zones and then ensure within the zone machines are split out so that any failure of a machine does not break the functionality of the rest. For example, lets say the ‘My Accounts’ feature runs over 20% of the hardware in a zone, and within that zone across 10 machines then the 10 machines must not all be on the same physical servers but should be in effect striped across the zone. Each zone is a separate fault domain, i.e. a hardware or performance issue in one zone cannot impact another functional zone. Within each zone their are also service policies for example a dev can create machines in a low security zone so they automatically get full admin access. For QA, the machines are created in a medium security zone and Prod is in a high security zone.
Simply your technology, processes and organisation to suit the cloud, and do not abstract away completely the underlying physical hardware from your application features.
What is the (private) Cloud? according to thetrainline.com
Although we at thetrainline.com were not exhibiting at the Expo this year, we are also making use of cloud technology and approaches: we blogged recently on why we built an on-premise private cloud for software test and integration.
The cloud has been an overloaded term and will continue to be so, but it is not just marketing hype or old technology just re-badged. There is still a wide range of cloud services and this is unlikely to reduce but continue to expand. The cloud ranges from complete solutions like Azure, AWS and RackSpace (a few of the key players), to hardware and software technologies that can be used to create your own private or hybrid cloud. The cloud is not for every business or for everything a business problem but over the next 5 years it is inevitable that more and more organisations from small to large enterprises will make use of the cloud. The cost of using Azure, AWS or RackSpace will continue to reduce and ultimately they will provide a compelling business case, not just for proof of concepts or development / test environments but for production systems.
Cloud Expo Europe 2013 provided another invaluable insight into how far cloud thinking and implementations have moved in the past 3 years. The ability to automate the creation of infrastructure is still in its infancy but will be the key that unlocks the cloud for many.