Trendspotting: Reflections on Big Data London 2022

Big Data London 2022 attracted 1000s of data professionals to London to keep up to date with new technologies and the latest shifts in the world of data.

The topics in the keynote sessions ranged from Data Meshes, to the rise of the Data Engineer – lots of related topics in the organisational paradigm shift utilising modern data management, analytics and domain data ownership.

With domains as owners of the data, specific business functions working with a data engineer to manage their own data, requires a modern analytics stack.

Benoit Dageville, snowflake founder, said during the great data debate:

Put data at the centre of your strategy, be really data-driven and that starts with having one single platform where you have all your data, where you make decisions.

If you want to build complex ml models and and and you do want to do complex analytics you have to combine this data and this is not only your data, it is coming from outside of your organisation. From the Covid pandemic, companies wanted to understand their data and had to look outside.

 

Benoit mentions having a single platform to store your data, a necessity for a Data Mesh, where data can be combined and analysed within a single platform, as well as shared with other companies using the same platform. Within the data mesh, it is critical to ensure that through a single platform, all users have secure access to data relevant to their domain.

Aligning with the Data Mesh paradigm and its secure access requirements, Astrato offers passthrough authentication for both Snowflake and Google BigQuery to ensure that each user will only see what they need to. Astrato does this in a no-fuss plug-and-play fashion where no additional setup is required, offering a seamless experience for users of Snowflake and Google BigQuery.

The rise of the data engineer acknowledges the 30% rise of Data Engineering roles over the last 5 years. There is a greater workload for engineering teams, with the ever-growing demand for new analytical views. Astrato is perfect for data engineers since it doesn’t add to their workload.

Role and user-level rules set in the data warehouse around data masking and row-level security are respected, access to table and views too. With primary and foreign keys defined in the data warehouse, data engineers no longer need to make room for or prepare analytical views – Astrato connects directly to tables and views as a user and intelligently suggest joins between tables. With a solution for fan traps and chasm traps looming, data engineers will no longer be haunted by complex BI demands.

We’re excited about the shifts happening in data and to offer a simpler, reimagined BI. If you didn’t see us at Big Data London this year, try Astrato today to see what’s new. It’s free for up to 5 users, with no time trial!