CFOtech Australia logo
Technology news for Australian CFOs and financial decision-makers
Story image

Big data projects, cloud migrations, and digital transformation: why it doesn’t need to be so hard

By Contributor
Fri 11 Feb 2022

Article by SnapLogic A/NZ regional manager, James Campbell.

“The pandemic has sped up digital transformation” is the phrase on everyone’s lips since the pandemic began. But the reality may be slightly different. Although it’s true that, in many cases, digital transformation is steaming ahead, some enterprises still find themselves hesitant to embrace the cloud despite the evident benefits, especially for moving big data projects. 

When the Australian Bureau of Statistics released its annual Characteristics of Australian Business Survey in June 2021, the results revealed that barely more than half (55%) of all businesses reported using cloud computing in 2019-20. Similarly, across the pond, the European Union’s statistical office surprisingly found that cloud computing has yet to go mainstream in the EU, with only 41% of enterprises using cloud computing in 2021.

So while the pandemic appears to have forced many enterprises to take a hard look at their business processes and technology infrastructure, there’s still much to be done to see cloud and digital transformation progress get closer to 100%.

So, why is it really time to move data projects to the cloud, what’s holding enterprises back, and how can they make it simple?

Flying to the clouds – or falling to the ground?

Put simply, by migrating their big data architecture to the cloud, enterprises can reap a number of benefits: from driving business growth whilst lowering the overall cost of operations, to increasing data governance and having a fast, scalable solution.

Moving big data to the cloud sounds simple enough. Still, it does require a high level of technical knowledge and often continuous coding resources from data engineers and core IT groups. This is why some enterprises either try and fail; or continuously postpone these projects. 

For example, many developers write code to integrate with each application’s programming interface (API) and authentication mechanisms. While this enables the data to freely move between various applications and a cloud data warehouse or data lake, it is time-consuming and often error-prone. These pitfalls are only emphasised during the maintenance stage of cloud-based big data projects.

As with any other software project, code decays over time and must be updated. Furthermore, if the developer who wrote the code leaves the company, often the IT organisation’s ability to understand the pipeline that is being used at the code level also vanishes.

One of the biggest challenges enterprises have had to overcome in moving to cloud-based big data projects is this time drain on IT staff. Finding individuals with the necessary skills and experience to build big data and cloud pipelines is a challenge.

Unsurprisingly, this is further impacted by the ever-growing skills gap in the IT landscape, an issue compounded to some degree by the ‘Great Resignation’ spurred on by the pandemic. The Australian Bureau of Statistics estimates that over 600,000 Australians expect to move jobs in 2022. That’s roughly 5% of the total workforce. 

These talented individuals are in ever shorter supply, and the demand is only increasing. If you do manage to get them into your IT team, having them focused solely on managing and maintaining the plumbing that supports their big data environment, both pre, during, and post-migration to the cloud is not a smart use of resources. It also has a big impact on another big issue in moving to the cloud - cost. 

With a team full of highly skilled individuals, you want them to have the time to focus on projects that deliver significant and strategic benefits to the business. The cloud provides flexibility and scalability, which can fuel innovation in the enterprise. However, the proposed time-to-innovation identified at the start of the cloud migration will never be achieved if teams are too busy focusing on infrastructure management to make the big data project work.

Buy vs build 

Finding a solution to this problem could seem tricky, but it is relatively simple, and it comes down to buying vs building. The chances of you needing to self-build every aspect of your IT estate is limited and, for most, cost-prohibitive, so why not look to purpose-built off-the-shelf SaaS platforms?  

If enterprises want to see their big data projects flourish in the cloud sooner, they should look towards modernising their data architecture. This includes introducing data integration (iPaaS), processing (BDaaS) and storage (SaaS) to the enterprise. This should enable organisations to seamlessly deliver large data sets to and from their cloud-based data lakes, regardless of where the data is coming from. 

An additional benefit of this approach is that it can also increase productivity by eliminating mundane, repetitive manual tasks involving adding information and transforming data, allowing IT teams to free up more time and focus on value-adding activities instead. 

Say goodbye to complexities 

Running big data projects in the cloud should be simple. All organisations, regardless of size, should be able to realise all the benefits the cloud provides as soon as they get up and running. It’s only in taking a step back at the planning stage and removing the complexities surrounding cloud migration and integration that businesses will finally be able to unleash their big data projects for innovation and to deliver business value. 

Article by SnapLogic A/NZ regional manager, James Campbell.

Related stories
Top stories
Story image
Artificial Intelligence
Frost & Sullivan recognises Genesys as leader in new reports
Frost & Sullivan has recognised Genesys as a leader in the cloud contact centre market for its robust cloud and digital capabilities.
Story image
Cybersecurity
Cybersecurity prompts upgrade for 1.3 billion electricity meters
ABI Research finds Advanced Metering Infrastructure (AMI) and cybersecurity concerns are prompting the upgrade of 1.3 billion electricity meters by 2027.
Story image
Digital Marketing
Similarweb acquires SEO and rank tracking company Rank Ranger
Digital intelligence company Similarweb, which specialises in analysing web traffic, has acquired Rank Ranger, a market leader in SEO and rank tracking.
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
SaaS
Atturra partners with Focus HQ to support Aus organisations
Atturra has executed a partnership agreement with Focus HQ, to resell and support the company's Australian developed SaaS-based portfolio management platform.
Story image
Telstra
Telstra, Google and Accenture launch 5G AR experience for AFL
Telstra, Google and Accenture are developing a new 5G powered augmented reality (AR) experience at Melbourne's Marvel Stadium for the footy season.
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Microsoft
Microsoft unveils adaptive accessories for disability access
Microsoft is introducing an expansive Inclusive Tech Lab to give people with disabilities greater access to technology through new software features and adaptive accessories.
Story image
Cybersecurity
Hard numbers: Why ambiguity in cybersecurity no longer adds up
As cybersecurity costs and risks continue to escalate, CEOs continue to struggle with what their investment in cyber protection buys. Getting rid of ambiguity becomes necessary.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Mobility
Hands-on review: STM laptop bags
The advent of hybrid working has meant we need laptop bags. We got our hands on two of the most popular laptop bags from STM.
Story image
Artificial Intelligence
Clear Dynamics closes $35M funding round, invests in global growth
The funding is a major milestone and speaks to Clear Dynamics’ vision for AI-enabled ‘composable’ enterprise software, the company states.
Story image
Manufacturing
HINDSITE wins Aerospace Xelerated Pitch Challenge with solution to support Boeing
Brisbane-based startup HINDSITE was the winner of the first ever Pitch Challenge organised by Aerospace Xelerated in partnership with Queensland XR Hub. 
Story image
Remote Working
IT teams deploy powerful technologies to enable remote work
"We found that IT teams mastered the challenges of remote work last year in large part by employing powerful yet easy-to-use technologies."
Story image
Tech job moves
Tech job moves - Datacom, Micro Focus, SnapLogic and VMware
We round up all job appointments from May 6-12, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Customer experience
Research unveils precarious customer loyalty for retailers
New research has found customers are reassessing established brand loyalties as their priorities and behaviours shift.
Story image
Remote Working
How organisations can meet employees' changing expectations
The global employment market has shifted dramatically in favour of employees, sparking the so-called great resignation, in which people are leaving unsatisfying roles in search of greener pastures.
Story image
SAS
New SAS service overcomes subscription fatigue for media companies
SAS has launched SAS 360 Match which helps media companies move towards a AVOD model to generate revenue as subscribers cancel.
The Access Group
Struggling to understand which transformative technologies will help your business? The Access Group provides a look into key opportunities and impacts for finance.
Link image
Story image
Artificial Intelligence
SAS unveils AI experience to improve kids' batting abilities
SAS has created The Batting Lab, an interactive experience using AI, computer vision and IoT analytics to help kids improve their baseball and softball swings.
Story image
Attain
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
Talend
Talend introduces new data health solutions for businesses
Talend has announced its latest version of Talend Data Fabric, with the release of Talend Trust Score enabling data teams to establish a foundation for data health.
Story image
Artificial Intelligence
SAS launches human-focused responsible innovation initiative
SAS has launched a responsible innovation initiative, furthering its commitment to equity and putting people first.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Microsoft
SAS Viya on Microsoft Azure to deliver 204% return - study
The Forrester Total Economic Impact study finds SAS Viya on Microsoft Azure brings a 204% return on investment over three years.
Story image
Digital Transformation
Unlocking the next digital frontier for educational institutions
Understanding where to invest in technology can be challenging for education institutions, especially after the COVID-19 disruptions.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Ransomware
Alarming surge in Conti Ransomware Group activity - report
A new report has identified a 7.6 per cent increase in the number of vulnerabilities tied to ransomware in Q1 2022.
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
Safety
Voxel hits total funding of $18M following ongoing wins
Since raising its seed round in September, Voxel has grown at pace, by decreasing on-site injuries by upwards of 80% and increasing operational productivity.
Story image
Fintech
Airwallex launches new bank feed integration with NetSuite
Airwallex has launched a new bank feed integration with NetSuite, developed in partnership with NetSuite solution partner, Onlineone.
Story image
Infrastructure
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
The Access Group
Increasing headcount isn't always the best way to grow. A good financial strategy can help solve many issues, and The Access Group shares the secret to success.
Link image
Story image
VPN
The most common online scams in Australia
No one is safe from online scammers, and many of these scammers have capitalised on the pandemic, using this confusing time to attack more people than ever.
Story image
Testing
Google and CSIRO use AI to help protect the Great Barrier Reef
Google has partnered with CSIRO in Australia to implement AI solutions that help protect the Great Barrier Reef.
Story image
SaaS
Cloudflare launches instant serverless database for dev teams
"Today we’re announcing our first serverless database which we expect will quickly become one of the largest databases in the world."
Story image
Remote Working
Australia’s remote workers face connectivity and security issues
SOTI's new report finds better video conferencing technology and improved security measures are top concerns for remote workers in Australia.
Story image
Adyen
Adyen expands partnership with Afterpay as BNPL payments increase
Adyen has expanded its partnership with AfterPay allowing more of Adyen’s merchants in more countries worldwide to use the BNPL provider.
Story image
Wasabi Technologies
Wasabi opens new cloud storage in Australia with Equinix
Wasabi Technologies has opened a new hot cloud storage region in Sydney, Australia, using Equinix services. This is the company's 12th global storage region.
The Access Group
Health and social care organisations are currently under significant financial pressure. Find out how financial transformation can help provide an effective route forward.
Link image
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image