Big Data: Five key challenges for companies

Data processing is a central link to economic development for companies these days. Their directors have understood this well and Big Data projects have multiplied greatly in the last several years. Although they partly justify this popularity, technology maturity and performances are far from being the only success factors of such projects.

 

Beyond the technical challenge…

It is crucial to understand that the success of a Big Data project does not only depend on resolving the technical and technological challenge naturally imposed by it. Feedback from our teams have allowed us to identify other different decisive points requiring special attention and real anticipation for companies:

Big Data

MANAGE THE DATA LIFE CYCLE

The implementation of Data Life Cycle Management (DLM) was already very important before the emergence of large data sets and related processing technologies. However, in view of the quantities and heterogeneity of the data manipulated by Big Data systems, this approach has become particularly critical. Not taking it into account would ultimately be a definite loss of time and money.

Concretely, that consists of defining and applying strategies and processes which will guarantee proper data routing, storage and processing during their entire lifespan in company’s information systems (from their creation until their deletion or archival).

Different frameworks of reference exist in order to guide companies in their DLM approach. The important thing is to really understand its main ins and outs in order to better adapt them to the activities and needs of the company (hence the importance of concrete and proven feedback and use cases).

 

CREATE VALUE FROM DATA

Data value creation is an economic and/or competitive advantage. It’s the main objective when setting up Big Data architecture. For more information, click here.

 

CONFORM WITH ETHICAL AND LEGAL REGULATIONS ON DATA (PROPERTY, PRIVACY)

There obviously exists a legal and ethical framework regarding data manipulation, which concerns more personal data (any information allowing for the identification of a physical person). It delimits how data can or cannot be used in terms of storage, analysis, sales, etc. and defines the applicable sanctions in case of non-compliance with these rules.

Since 25 May 2018, the General Data Protection Regulation (GDPR) oversees the personal use of data by companies and defines individual rights. If a company breaches this regulation, it can incur a fine of up to 4% of its turnover. In France, the CNIL (Commission Nationale de l’Informatique et des Libertés, or the National Commission on Informatics and Liberty) is responsible for applying this regulation. It plays a role of surveillance, control and punishment when violations are observed, but it also provides information and advice to businesses and individuals.

 

ENSURE CYBER-SECURITY

Protecting one’s data against cyber-attacks is obviously of paramount importance for the company. The risks in the case where it would be victim of such an attack are plentiful: direct financial losses (costs of a technical investigation, bringing infrastructure into compliance, etc.), image deterioration, indirect financial losses (fines, loss of client contracts, etc.) or even loss of competitive advantage (theft of strategic data, industrial espionage).

Although cyber-attacks have not appeared with Big Data, it does seem that its emergence has had a quasi-contradictory double effect. The increase of data sources, of the quantity of data processed and of actors that make up the Big Data ecosystem creates an increase of potential flaws. However, the rise in Big Data technologies has allowed for the development of more predictive and adaptive approaches (Machine Learning, predictive analysis, etc.) as well as more effective data cross-checking in real-time and thus the possibility of better defending oneself when facing this kind of attack.

 

MASTER NEW BIG DATA PROFESSIONS

The Big Data technology gap has pushed existing data processing occupations to evolve and for new ones to develop: Data Analyst, Data Scientist, Data Engineer, Data Architect, Data Security Officer, Data Protection Officer (which has become mandatory by the GDPR), Data Steward, etc.
It is very important for companies to master these positions and the skills inherent to them, in order to be able to implement the organisation necessary to the success of their data value creation projects.

 

Let's talk about your projects together

bouton-contact-en