Advent of data

4 min read

Data Anarchy

Joker (2019)

We live in a data-rich world, and a state of anarchy — Data is everywhere and with it comes the need as a data analytics team to understand how we’ll collect and use it because when it comes to business intelligence, there are always tradeoffs between the needs within a business, the requirements of the company as a whole and individual daily uses.

I’m Majdi, a 10+ years senior developer, former entrepreneur and now Head of Data at which handles hundreds of millions of data entries per week. Our data platform is 3 months old and as you can imagine being a scale up and having a pretty crazy growth let us doing everything we can to avoid anarchy. Spoiler alert we made it, here's my feedback.

Good Bye, dictatorship!

As Head of Data, I had the chance to built the data department from the ground and we start building our data platform 6 months ago with modern stack foundation thanks to one passionate data engineer. All data governance knowledge are discussed from a while but it's always an interesting part of a company challenge from its DNA to the technology used through employees comprehension.

In my own opinion, the first question that everyone should answer is around a well defined data vision – how to decentralize everything and avoid IT restrictions because let's face it once and for all data shouldn’t be concentrated in the hands of a few.

Simple answer but complicated implementation depending on the situation of the company. The focus here is about choosing the right tools and define the principle of least privilege to let every person in a company accountable for their part in creating and using data effectively, efficiently and responsibly.

Note that open source hate dictatorships and that's pretty cool because the open-source data ecosystem is mature and industrial!

Avoid silos and anarchy

Playtime (1967)

Now that we defined our vision and have the foundation, the next step is much more complicated because is about not creating conditions of toxic data silo culture which leads to the beginning of anarchy and the worst case-scenario for any company.

Data must be held accountable and must hold the truth.

The challenge is that data may not always speak for itself so we need to adapt and avoid the scenario when data is not equal, fed by uncoordinated silos because the result of inconsistency leads to wasted time and money and more than that lack of IT trust.

We must accept the facts that today, anyone can compute its own output, resulting in inconsistent data and duplicated efforts. The growing tech ecosystem is incredible but doesn't help with all data-oriented saas tools available on 1 click that come out every week, all no-code tools that would make developers blush and the ease we have today of interconnecting data sources..

Not so easy to handle but one of the highest priority as data custodian !

The trouble of anarchy is knocking on every door and the trouble with a data anarchy is that it can cause delays and add friction during the decision-making process. What do we need to do to avoid people having uncontrolled new data layer between business and data intelligence?

Let’s talk about democracy

Avatar (2009)

Data is no longer just data. Data is only a part of the experience and we have to built something more on top of it. Too often, people build their data experience around the tools they want to use. Spoiler alert there are too much tools and you’ll be overwhelmed by all the tools on the market – let's focus on 3 major notions:

Data discovery

The key practice for a company is to empowers users to find, understand and use data across all applications effortlessly. With a unified user experience, data discovery becomes an organic part of the business.

Company fellows

It's not just about the individual. We must create a space for people to be a part of a global engagement. People must understand that global effort create greater value. The role of any data team is to support fellow in data thinking and implementation. It's an essential educational role that help having a real data-driven culture.

Data health

Having the right data, in the right place at the right time is key to making smart business decisions fast — a healthy data system is one that is flexible, highly secure and easily accessible by the entire company. This includes up-to-date, accurate information about the business, the product and all customers as well as internal data on employees and business processes.

And now?

Any form of democracy need continuous improvement and requires updated tradeoffs between different groups. That's also related to the business intelligence systems which become more comprehensive and complex by the day.

The key to success is to keep in mind that it’s always about the business needs, not the technology platform because analytics is the process of using data to make better decisions and support business processes — in a nutshell, analytics means using data to learn something new about an unknown subject. This could be within your own company or industry or on a global level.

Focus on Data Experience and keep asking questions about the data. This will guide the stakeholder in data democratization and often reduce the work of by an order of magnitude.

Joyeux Noël 🎅🏽