Welcome! This is the ninth edition of CloudProfs, sent to subscribers on September 24. See the email in-browser here.
If you enjoyed this newsletter, why not sign up to receive it in your inbox every week? Or if you have any feedback, email the editor.
What’s Been Said and Done in Cloud This Week
European cloud providers have collectively more than doubled their cloud revenues in four years, but have lost out in market share. This is the finding from Synergy Research, whose data show that the European cloud market has hit €7.3bn (US$8.8bn) in the second quarter of 2021. Deutsche Telekom is the European leader, accounting for 2% of the market, followed by OVHcloud, SAP and Orange. Yet the total market share, of 16%, is well below the 27% of 2017. This, unsurprisingly, is due to squeeze from the hyperscale providers. Some of the highest growth in Europe is seen in platform as a service (PaaS) with database, IoT and analytics services. Source.
Amazon Web Services is opening a new cloud region in Auckland, New Zealand. The prospective timeframe is 2024, so a long time to wait, but the region will have three availability zones and is set to create 1,000 new jobs, per an economic impact study released as part of the launch. Other AWS regions in the works include Australia, India, Indonesia, Israel, Spain, Switzerland, and the United States. Source.
Secure Access Service Edge (SASE) is expected to hit double-digit billion-dollar revenue by 2025, according to the latest analysis from Dell’Oro Group. SASE is defined by Gartner as ‘combining network security functions with WAN capabilities (i.e. SD-WAN) to support the dynamic secure access needs of organizations… delivered primarily aaS and based upon the identity of the entity, real-time context and security/compliance policies.’ Dell’Oro Group noted that enterprise interest in SASE was ‘skyrocketing’ as cloud-first and mobile-friendly architectures became the norm. There are now 35 SASE vendors, according to the analyst firm. Source.
BONUS: McKinsey has released an interactive cloud cost-optimization simulator. The simulator details the range of levers that can be used to substantially reduce costs for one illustrative scenario of an application on the cloud. See it here.
Onboarding an Organization for a Fully Cloud Strategy
Read another article by Sjoukje – An Overview of Azure Containers – which appeared in CloudProfs #6.
Technology is not only improving people’s lives but is also making work easier in many sectors. On top of that, we have all witnessed how the pandemic has accelerated digital transformation. Launching technology-driven initiatives to enable remote work and distance learning, online shopping and more. For most organizations, this shift was not possible without cloud technology.
Cloud is here to stay, and to stay ahead of competition and on top of customer demands, more organizations need to shift to cloud computing. But the only way to truly succeed with cloud, is by embracing it fully across your entire organization. And this is more then just technology. This also means that people and processes need to align to the strategy as well.
In the next sections, we are going to look at some important aspects that make up a successful cloud strategy.
Preparing and training IT staff for the cloud
The first important step is to prepare ant train your IT staff. They need to function as change agents for supporting and emerging cloud technologies. The staff needs to have a clear understanding of both their roles and any changes to their current position, and the business case behind it, and they need time and resources to explore the new cloud technologies. This team will need to lead the adoption of cloud services and help the organization understand and embrace all necessary changes.
Establish a Cloud Center of Excellence
Every business nowadays needs to drive digital transformation to stay ahead of competition. This requires constantly evolving software to keep up with competitors. Therefore, Agile development and DevOps practices need to be in place. DevOps needs immediate availability of infrastructure in place for deployment. And therefore, PaaS and containers are well suited. This makes it very important to the cloud strategy that continuous delivery is well implemented. And one way to establish this is by implementing a Cloud Center of Excellence, which is a control centre which the IT department can implement, supervise and manage its cloud projects operationally. It is responsible for implementing and managing security, compliance, performance, and cost control in a (fully) automatic way.
Cloud enabled applications
To fully opt-in for cloud, prioritize cloud migrations for applications that are heavily used in your organization. This will not result in cost-reduction in the beginning, however in the long end it will be cost effective. It enables flexibility and scalability for apps with variable usage. It also enables frequent deployments for these applications. Customer feedback and changes can be incorporated at a rapid pace, which will lead to higher agility and customer satisfaction. For all new applications, a cloud first strategy should be the approach.
Establish a cloud culture
This last topic is in my opinion the most important one in being successful in cloud. And unfortunately, this is overlooked at many organizations. Not only the IT staff needs to be trained and shifted to a cloud centric approach. When you do a cloud transformation, is most cases it is technology that is the biggest challenge. It is the people and the culture. And this goes beyond your IT staff and the professionals that run the Cloud Center of Excellence. The whole leadership must be in and support and drive these changes in the organization. You need some champions that advocate the new way of working, but also recruit new passionate and skilled people. Continuously measuring progress and outcomes must be a top priority. And don’t be afraid to fail and change the approach if needed. Changing the way people think and work, is the most challenging part of cloud adoption.
DevSecOps Insight: What Tech Resonates Most With CNCF Members?
The Cloud Native Computing Foundation (CNCF) published its latest End User Technology Radar this week, a guide to a set of emerging technologies based on the experience of its community. This radar’s theme focused around DevSecOps, with technologies placed in the Assess, Trial, or Adopt phase. A total of 16 technologies were analysed, with eight being placed in the Adopt phase, one in Trial, and seven in Assess.
This article aims to look at some of the potentially lesser-known technologies and projects which made the cut.
ArgoCD (Adopt). ArgoCD is a declarative GitOps continuous delivery tool for Kubernetes. It follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. It is implemented as a Kubernetes controller which continuously monitors running applications and compares the current, live state against the desired target state – as specified in the Git repo. At time of print, a total of 161 organisations were officially using ArgoCD, including BMW Group, Capital One and Electronic Arts.
Artifactory (Adopt). Artifactory, from JFrog, is marketed as the world’s first universal artifact repository manager. Its function is as the single source of truth for all packages, container images, and Helm charts, as they move across the entire DevOps pipeline. It promises to increase development speed across any environment by automating and managing binaries and artifacts and integrates with a user’s ecosystem and DevOps tools. It was the third most popular tool on the CNCF ranking, behind Terraform and Hashicorp Vault.
Open Policy Agent (Adopt). The Open Policy Agent (OPA) is an open source, general purpose policy engine which unifies policy enforcement across the stack, providing a high-level declarative language which lets users specify policy as code and simple APIs to offload policy decision making from software. This declarative language is Rego, which is purpose-built for expressing policies over complex hierarchical data structures. OPA can be used as an admission controller in Kubernetes, an external authorizer with Envoy, as well as validating Terraform plans.
Trivy (Assess). Trivy, from Aqua Security, is portrayed as a comprehensive open source vulnerability scanner for container images. The product has a compact database with auto-update capabilities that do not require middleware or database dependencies. Scans take seconds with critical CVEs being filtered directly in the command line. Users can view results of scans directly in the GitHub UI via the GitHub Action, or in any Kubernetes dashboard, using Aqua Starboard.
Xray (Trial). Xray is portrayed as a complete test management tool for Jira. It supports across the entire testing lifecycle, from planning, to design, to execution, to reporting. The app supports manual and automated tests, and its features include seamless integration with Jira and Jira concepts, as well as generating advanced reporting with eazyBI.
While half of these tools have been considered worthy of adoption, there was a caveat. The Radar report summarised three themes from the products analysed:
- Security is the main focus of DevSecOps at the expense of developer experience. The DevSecOps developer experience was considered ‘cumbersome’ with hard-to-consume tools creating a messy development environment
- The pace of change in the security space is rapid. As more tools come to the market, practitioners are struggling to evaluate what best fits their needs
- Microsegmentation capability is very important but has a significant challenge. While many organizations have introduced robust DevOps processes, traditional network security practices have not kept pace with the shift to cloud native
The full list of tools analysed was, in order of popularity: Terraform, Hashicorp Vault, Artifactory, Sonarqube, Calico/Tigera, ArgoCD, OPA, Istio, Sonatype Nexus, GitHub Actions, Cilium, Harness, Linkerd, Hashicorp Sentinel, Trivy, Xray.
Tutorials: Test Your Bicep Strength
If you are interested in Bicep, the domain-specific language for deploying Azure resources declaratively, Toon Vanhoutte, author of Your Azure Coach, has put together a couple of tips and walkthroughs this month.
Validating a Bicep file in YAML pipeline
While there is no need to compile files in infrastructure as code (IaC) templates, there is value in validating a Bicep file during the build. This can be done through the az bicep build command.
Because it is an az command, you would typically run it via an Azure CLI task. I am not in favor of that, because I don’t want to link my build stage to a certain Azure subscription (unless I would like to execute my deployment already to a dedicated build environment).
That’s why I think it’s much cleaner to use the PowerShell task. In this way, it’s a 100% clear that this is a pure syntactical validation, without any link to Azure. Here’s an example:
– task: PowerShell@2
displayName: Validate Bicep file
inputs:
targetType: inline
script: ‘az bicep build –file infra.bicep’
workingDirectory: ‘$(Build.SourcesDirectory)/infra’
When executing this code in an Azure DevOps pipeline, with a Bicep file which points to a non-existing module, the exception ‘An error occurred reading file’ appears.
Conditional deployments with Bicep
This involves using Bicep to roll out Azure API management throughout different environments. Two types of conditional deployments had to be used; as in the development and test environment, API management was used without network integration. In acceptance and production environments, it is deployed internally to the virtual network.
Two types of conditional syntax, based on C#, as Vanhoutte writes:
Conditional deployment of a resource. Based on the virtual network type, I can determine if I should provision an Application Gateway or not. This can be easily done through the following syntax:
resource applicationGateway ‘Microsoft.Network/applicationGateways@2021-02-01’ = if(apimVirtualNetworkType == ‘Internal’) {
name: applicationGatewayName
…
}
Conditional configuration of properties. Within my API management resource, the virtualNetworkConfiguration section should not be included when API management will not be integrated with a virtual network. To achieve that, we can use a conditional expression.
//Create API Management Service
resource apiManagement ‘Microsoft.ApiManagement/service@2020-12-01’ = {
name: apimName
location: location
…
properties: {
…
virtualNetworkConfiguration: (apimVirtualNetworkType == ‘None’) ? null : {
subnetResourceId: resourceId(apimVirtualNetworkResourceGroup, ‘Microsoft.Network/virtualNetworks/subnets’, apimVirtualNetworkName, apimVirtualNetworkSubnetName)
}
virtualNetworkType: apimVirtualNetworkType
}
}
Read Vanhoutte’s excellent blog here.
Hidden Gems and Secret Knowledge
A cool selection of recent (or recently updated) cloud repositories and tools across vendors and languages. Got a tip or are you working on a project you want the world to know about? Email the editor today!
100DaysOfCloud (2): Daily log of Steven, an aspiring cloud engineer looking to learn and grow in the environment. Latest update day 9. (Not dissimilar to John Breth in CloudProfs #8).
collie-cli: allows users to manage AWS, Azure & GCP cloud landscape through a single view. New release: v0.8.0 (Sep 23). Primary language: TypeScript (98.5%)
cosign: Container signing, verification, and storage in an OCI registry. New release: v1.2.1 (Sep 20). Primary language: Go (96.3%)
IBM-ROKS Toolkit: Set of tools and files that enables running OpenShift 4.x on IBM Public Cloud in a hyperscale manner with many control planes hosted on a central management cluster. New release: v4.6 20210923 (Sep 23). Primary language: Go (95.5%)
nubesgen: Easily generate Terraform templates and GitHub Actions for your project. Currently in technology preview. New release: v0.5.1 (Sep 24)
BONUS RESOURCE! The Cloud Native Community Cookbook volume 1.0. This is not your typical tech cookbook! From Equinix Metal, prominent cloud-native faces have shared their favorite recipes from around the world, including Lebanese spinach pies, sauerbraten, and roasted cauliflower salad. (Direct to PDF, no signup required)