Data integration and its role in a multi-cloud environment

The cloud market in Australia reached the $1 billion mark for the first time in 2020, mainly due to COVID-19 related increases in collaboration apps and other work-from-home solutions.
Cloud has cemented itself as the platform powering technology in the 'new normal', providing an avenue for business continuity and for people to remain connected despite city lockdowns and stay-at-home workplace conditions.

With an increasing focus on the seamless delivery of applications to a remote or distributed workforce, many organisations today use more than one cloud and storage service. In a single network architecture, they may tap on two or more different cloud service providers like Amazon Web Services (AWS), Google Cloud Platform, IBM Cloud, and Microsoft Azure.

Research from Australian analyst Telsyte have revealed that AWS has the largest share of the market in Australia, followed by Microsoft's Azure. In particular, collaboration tools and video conferencing, which are built on cloud platforms, have seen the biggest uptick in usage.

Cloud platforms are all unique, providing different architectures, accessibility and applications. They also differ in the way they offer support to customers, which means that assessing and controlling the benefits, cost, and level of security can be complex for organisations taking a multi-cloud approach.

Organisations in Australia have demonstrated a propensity to opt for best-of-breed applications in a multi-cloud architecture, and not necessarily lock themselves down to a single provider. Organisations are also willing to take on associated security risks, and invest effort and time to properly manage and integrate different services from different cloud providers.

Are we in danger of single-vendor lock in?

Major technology vendors like Microsoft, Oracle and SAP have long been emphasising the benefits of a single integrated architecture, encouraging customers to run applications from a single vendor across operations like Accounting - Finance, Human Resources or Sales and Marketing.

No doubt, having various applications running on a single integrated technology platform for the purpose of interconnectivity has its benefits. Organisations may choose to onboard customers in a Customer Relationship Management (CRM) application that is linked to the finance application that they are already using to ensure the same customers pay for products or services in a timely manner. This also takes away the steps of creating two separate sets of customer records in two different applications.

However, IT leaders have always been careful of vendor lock in. Instead, many prefer to acquire applications from different vendors that can best meet requirements and choose to manually integrate them using data integration technologies.

For instance, manufacturing businesses may adopt SAP applications for manufacturing processes and PeopleSoft solutions for human resource functions. Retail businesses may choose to adopt Oracle applications for procurement and supply chain, but use Siebel CRM solutions to support sales and marketing functions. These same businesses may be using Microsoft's technologies too for databases and business productivity applications.

Simply put, most organisations tend to opt for a multi-compute architecture.

First multi-compute and now multi-cloud

The multi-cloud approach took some time to become somewhat of a norm as well, just like the adoption of multi-compute architecture.

Organisations only started to seriously migrate their data and major workloads from their data centers to the cloud when they saw the benefits of more flexibility and lower operational costs. Gradually, organisations have come to rely on the cloud for much of their IT infrastructure setup. Also, IT teams can focus on driving more strategic technology roadmaps through cloud, and relinquish the stuff up have been working on/things running properly.

Most organisations began by running part of their applications on cloud and the remaining on-premise when they first experimented with cloud computing. In other words, they took a hybrid cloud approach, operating cloud computing environments that combine on-premise, private cloud, and third-party public cloud.

Multi-cloud is simply an extension of that approach whereby organisations run some of their applications on-premise and others on different cloud environments, tapping on services from different public cloud service providers.

Naturally, organisations selected cloud providers based on the applications they already have in place. Very likely, they chose Microsoft Azure cloud because they were using Microsoft Dynamics, Office 360, or SQL Server. Others might opt for Google Cloud Platform as they run data science applications, or they might go for AWS as they are using S3 storage.

At the same time, they would have continued to run some of their legacy applications on premise in their data centers.

It is not always rainbows and sunshine in multi-cloud

Most importantly, multi-cloud strategies allow organisations to customise their technology infrastructure in a way that suit their own unique business processes. It allows a company to benefit from the advantages that each platform brings and minimise the impact of their shortcomings.

Nonetheless, organisations may benefit from multi-cloud environments and gain support on governance, risk management, as well as compliance and regulations.

Multi-cloud architectures can bring challenges too. The most obvious is that data stored with one cloud service provider will be siloed. The issue that this means organisations will not be able to get one holistic view of their data across a multi-cloud environment.

It is incumbent upon the service provider to secure their environment, and for this reason cloud security at the vendor side is generally first rate. However, data security can be an issue when organisations have to access the disparate data across the different clouds. There is always the risk of exposing the data to unintended users.

Embracing heterogeneity in multi-cloud with data integration

Organisations may want to explore a robust cloud data integration solution powered by data virtualisation to truly embrace the strengths of multi-cloud. Data integration tools have reliably delivered integrated data from systems like Accounting and Payroll, CRM, Enterprise Resource Planning (ERP), etc. to specific destinations such as a data warehouse for years.

Currently, with the growth of cloud, data integration has also evolved to integrate data from on-premise and cloud systems. Modern data integration tools based on data virtualization can efficiently integrate data spread across public and private clouds, and delivering them in real-time to systems that either reside on-premises or within the cloud environment itself.

These data integration tools also come with strong security capabilities, ensuring that the data stays protected. Access to the data continues to be limited to the intended end users only in line with their approval and authentication levels.

Modern organisations have the options to choose where they store their data and run their applications, be it on-premises, in the cloud, in hybrid mode across both on-premise and cloud, or even across multiple cloud environments. Especially in the current pandemic-hit business landscape, organisations need the agility and flexibility to freely take an applications and data management approach that best suit their business requirements and not be restricted by concerns over the complexity or security risks of data management.