Bryan Balfe, CommVaultBryan Balfe, CommVault

Changes in decision-making tactics, business methodology, and privacy have all risen as a result of big data

Recent research by MGI and McKinsey’s Business Technology Office found that the amount of data in our world has been exploding, and analysing large data sets – so-called ‘big data’ – will become a key basis of , underpinning new waves of productivity growth, innovation, and consumer surplus.

As such, leaders in every sector are having to grapple with the implications of big data, and not simply data-oriented managers. The increasing volume and detail of information captured by enterprises, the rise of multimedia, social media, and the Internet of Things will fuel exponential growth in data for the foreseeable future.

“For many good reasons, South African organisations have been slow to adopt a data-centric view of managing data,” according to Bryan Balfe, channel manager at CommVault. “Furthermore, we still rarely see the data as the focal point for decision-making in data centre design and operations.”

He believes the traditional pillars of storage, networking, server (compute) and platform still tend to dominate planning and decision-making, which is preventing the innovation required to allow organisations to scale with the data explosion. “This is being witnessed in all sectors,” he says. “Many organisations are too busy managing the environment and keeping the ‘lights on’, which is causing a ‘can’t see the forest for the trees’ scenario. This is resulting in companies retaining far too much data than necessary, at massive cost, without a cohesive data management plan that is aligned to business requirements.”

For most South African organisations, ‘big data’ remains an emerging trend rather than a practical reality, according to , MD of Master Data Management. “However,” he says, “Some early adopters are currently making investments as they believe they will see benefits. Although there are a few early adopters, I am not aware of any current implementations that are sufficiently advanced to show measurable improvements as yet.”

According to , CIO of PBT Group, some businesses in SA are already successfully managing their business strategies using information, and others are fast starting to realise the value of information – and how the underlying data can be utilised in making key business decisions.

“Some businesses are also starting to pay attention to big data analytics. However, there is still much to be done to ensure that most companies understand what exactly big data is and, more importantly, how to use it,” he explains.

Big data is much more of a value augmenter than a new value generator, says Rennhackkamp, yet being able to derive this valuable information from the data within a business means that big data is able to achieve additional information-driven gains for a company.

According to Shaid Greeff, emerging technology architect: big data at T-Systems, it is without a doubt that the big data hype is here to stay and that in time, it will evolve into a mainstream requirement for businesses and organisations as the value of all data related to day to day activities become a standard requirement. “For years, business intelligence (BI) has been a top CIO priority and therefore big data will in time become part and parcel of information management activities.”

However, South African business is still slightly behind the rest of the world with respect to adoption of big data solutions, he says, and according to an Ovum report, this slight lag is usually due to a suboptimal infrastructural environment and the conservative nature of many South African business executives.


ThoughtWorks mentions a disturbing trend of businesses storing vast amounts of personal data unnecessarily in its recent research and recommends businesses adopt an attitude of “datensparsamkeit” (data economy) and store only the absolute minimum personal information from their customers.

According to Quintis Venter, developer and senior consultant at ThoughtWorks this trend is not a surprise so much as it is a logical progression of the realisation that there is a strong positive correlation between more specific customer data and higher conversion rates and profits for product and service campaigns. “SA is not unique in this respect, and it speaks to our current position in the developed vs. developing countries debate as we are not definitively either.”

Shaid Greeff, T-SystemsShaid Greeff, T-Systems

When it comes to privacy and data , says Venter, there is no silver bullet. “It is important for companies to realise privacy is usually not a binary choice. Seldom will we have to choose between capturing datum or maintain user privacy, as there are often ways to accomplish both.”

One big challenge SA is facing however, he continues, is the law regarding data and privacy in general from both a consumer and business point of view. “From a purely technological standpoint, SA can hold its own, although being a technologically “developed country” requires more than developed technology. The law faces the inherently difficult task of keeping pace with that of technological innovation, making legislation around privacy extremely difficult. Legal half-measures can only go so far.”

Even with the advent of legislation such as Protection of Personal Information (POPI) Act, SA still lags behind more mature markets in the enforcement of poor data management practices, explains Balfe. “SA has a legacy approach to data retention policies and no real external forces driving change. The cost of non-compliance or breaches is unclear, and the likelihood of punitive action is very unlikely. Therefore there has been no real reason to address this issue.”

Rennhackkamp believes South African companies are cautious when it comes to big data implementation because of privacy issues. “Big data service providers, and to a degree, the government, should pay close attention to issues like privacy here and put the necessary regulations in place, in order to appropriately govern the local utilisation, storage and analysis of big data, as privacy and data sharing will always remain areas of concern,” he says.

He also believes cost is a factor when it comes to embarking on big data analytics exercises. “Due to the rand-dollar exchange rates, the real degree of the value augmentation is still being somewhat questioned,” Rennhackkamp explains.

He states that globally, only the organisations that really work with data as their primary product are miles ahead in terms of big data – the examples being the eBays, s, Amazons, Googles and Yahoos of the world. “Yet for ‘normal’ bricks and mortar, as well as conventional on-line businesses, the opportunities exist that big data can provide some competitive advantage. This benefit should gain traction and be explored,” he says.

Allemann does not believe South African conditions are substantially different to any other regions with regards to the ability to leverage big data. “However, the relative lack of maturity around success factors such as data governance and data quality, combined with the general shortage of data management skills, can increase the risk of failure of big data projects when compared to more mature markets such as the USA.”

According to Greef, to fully cater for the requirements such as data privacy, governance and compliance and the like, a big data solution must form part of a holistic information management strategy. He believes the ecosystem of big data is constantly evolving and being developed to close the functionality gap which currently exist in traditional Relational Database Management System (RDBMS) environments.

“The big data platform can be regarded as a data platform which can act as a source into already existing BI and data warehousing (DW) platforms thereby allowing for the interoperability of these new and existing technologies,” he says.


It is clear that with certain technology trends, SA has adequately adopted technological advances, Greef continues. “As for big data, according to a hype cycle, big data in SA is in the early adopter phase, with a two to five year period before mainstream adoption.”

This history of BI and DW in SA provides the base on which a big data ecosystem can be phased into existing Information Management (BI and DW) environments, he explains.

“The challenge however, is the need for new set of skills to understand how to collect all the data, how to store it economically and how to extract it and extrapolate value from it, namely the unstructured data,” he says. “The foundation to support a big data ecosystem is in place, the skills are being developed. Are we there yet? Well, we are slowly developing towards it.”

Quintis Venter, ThoughtWorksQuintis Venter, ThoughtWorks

According to Allemann, the biggest challenge facing any organisation embarking on big data initiatives is the global shortage of data management skills. “For the South African market this could translate into a drain or pressure on existing resources as these skills are in demand globally. However, organisations could consider other mediums such as eLearning to improve the data management skills of their current local resources.”

Rennhackkamp agrees skills are a challenge locally, and also believes cost to be an obstacle to adoption.

“SA sits on the borderline here, in a way similar to Brazil. We have some skilled resources, but not enough to fulfil the growing demand. I would like to see more focus on formal educational programmes, like we see Masters and PhD level programmes in informatics, advanced analytics, information and data science overseas,” he says.

What is really encouraging however, Rennhackkamp says, is some organisations establishing locally-hosted cloud infrastructures, also available for public acquisition at locally affordable prices, which overcomes the network bandwidth and rand-dollar exchange issues. “One of the other fortunate aspects for developing countries is that many of the big data technologies, such as Hadoop, were designed to run on large clusters of inexpensive commodity hardware, often on open-source software. But I have to caution here – a platform like Hadoop is not necessarily always the appropriate solution for each and every big data business problem,” he says.

We often hear our region described as First World / Third World in its makeup, with many large organisations fitting into the size and scale of many first world companies and organisations, explains Balfe. “However, without the same focus on best practices pertaining to data management and data centre planning, many things have spiralled out of control including infrastructure, network connectivity and services costs. This in turn poses a challenge in breaking the cycle. As this problem gains impetus and becomes more difficult to manage, we see a tendency to focus on planning of the component parts, without a examining the data challenges driving the pressure on the environment. This in turn stifles innovation and sees the traditional three to five year hardware refresh dominating the planning process.”


When it comes to how SA companies take on the various challenges related to big data, such as capturing, curation, storage, searching, sharing, transfer, analysis and visualisation, there are many different ways in which local organisations are coping with the information growth challenges.

According to Greef, South African companies are in a fortunate position as there are many established local software and hardware vendors and service providers in SA who provide the technology and expertise to support the big data framework.

“Importantly, BI and DW together with information management techniques are already well entrenched and supported in the South African market,” he says. “Businesses in SA that have implemented DW and BI solutions already have information management methodologies in place to support the end-to-end process of information management.”

In this instance, he explains, the process would remain the same, the only new ingredient would be a new storage platform (such as Hadoop) to support the storage structured and unstructured.

Rennhackkamp believes many organisations ‘blindly’ want to jump on the big data bandwagon, without thoroughly investigating the impact and what their businesses actually need to have in place to effectively implement big data. “Many organisations think they need to instantiate or get access to some big data platform, such as Hadoop, without really investigating what it is going to add in business benefit,” he says.

He believes organisations need to first investigate what the business benefit would be, profile their existing data, as well as the big data that they will be able to access and process, determine if there are any integration points, and design upfront what they’re going to do with all this data.

“Organisations also need to study the available infrastructure and determine what’s feasible and possible for them. Often the cost of access to a platform is negligible in comparison to the on-going running costs, or up-and down-load tariffs. Having understood all this, businesses should then be equipped to take on the challenges as listed above, that they may be presented with, and in the long run obtain the identified business benefits that they set out to achieve,” Rennhackkamp says.

He also believes an important aspect is the ability to make sense of unstructured big data. “It takes a lot to disambiguate unstructured text – which is the bulk of most ‘normal’ organisations’ big data – and then analyse it as structured additional data in conjunction with its existing data. The other ability organisations need to build up is to visually analyse and drive insights out of large volumes of structured and unstructured data.”

Balfe explains that the CommVault view is aligned with the popular axiom of, “First rule of holes is once you know you are in one – stop digging. This means that whenever there are decisions to be made on future plans, it provides a great opportunity to stop, learn from the past and engage with all stakeholders in the organisation, not just IT.”

Furthermore, he says, collaboration and making subtle changes in the approach can have far reaching benefits for future growth. “The formation and development of data management and data retention policies should not rest solely within IT. It should include all other business stakeholders such as finance, legal, compliance and regulatory as well as commercial owners, allowing companies to build out a more rounded approach. Furthermore, other than meeting the requirements from a retention and storage policy perspective, organisations can also provide improved access to information which allows other parts of the organisation to thrive.”