Current vs Modern BI

The data world is changing and maturing. Is your Business Intelligence solution still fit for purpose?

With technological advances in the infrastructure, it should be easier to get more value from your data, right?  Matt Turck shares his view in his analysis of the current Machine Learning, AI and Data Landscape:

“Today, cloud data solutions (Snowflake, Amazon Redshift and Google BigQuery), lakehouses (Snowflake, Databricks) and data clouds (Snowflake) provide the ability to store massive amounts of data in a way that’s useful, not completely cost-prohibitive and doesn’t require an army of very technical people to maintain.

In other words, after all these years, it is now finally possible to store and process Big Data.”

But for many companies, there is a big resource gap. And the way their data teams are currently working with data is often a far cry from what modern BI should look like.

The current BI process

The BI development process is very labor-intensive.

Projects can take weeks or even months to complete depending on the volume of data, questions, and other requirements.

For each new business case, your data analytics team cycles through many steps to produce a lasting and impactful product.

The current BI development process is iterative and usually goes something like:

1. Identify the business need and define requirements. Usually, business requirements are identified in silos and intended to solve the latest data request from management.
2. Assess the current state of data sources, metrics, and tools to meet the business need.
3. Identify the gaps between the current state and desired business solution. Identify data sources, metrics, and tools the customer needs. Clean, define, join, and transform data for that business need.
4. Analyze the data through exploratory data analysis to identify trends, and key findings to incorporate into the final product.
5. Design user-friendly, intuitive, and actionable dashboards that solve the business problem.
6. Implement the BI solution by testing the product, deploying, and training users.

 

 

 

But what happens next?

Analytics teams then get caught in an infinite loop of solving individual business problems, which prevents them from focusing on strategic data initiatives.

So how can our data teams succeed in the modern data world?

Teams need the resources to produce BI solutions that all draw from a single source of truth.

A modern approach to BI

With legacy BI tools, the process is Extract, Transform, Load, Transform (ETLT). But a more modern approach, like cloud-first solutions, simplifies this process to Extract, Load, Transform (ELT).

Extract your data from the source (ERP or CRM for example), load your data into a staging area or cloud database in the original format with little to no processing, and transform it in the cloud data warehouse where it is governed and accessible for business users.

With data stored in a Cloud Data Warehouse (CDW), resolving bugs, adjusting data cleansing, and updating business logic in data transformations is definitely faster and easier.

The main benefit is keeping business logic close to the data.  BI developers can point to a single source of truth, where data cleansing and transformations are consistently maintained and updated.

Shifting to a CDW has many benefits. With cloud-native software these include:

• Improved data governance
• Reduce data silos
• Lower total cost of ownership
• Improved speed and performance
• Increased data storage
• Near-infinite scalability
• Better disaster recovery
• Native support for unstructured data
• Enhanced data sharing

Astrato’s cloud-native, live-query approach, means that your data is always up-to-date when you visualize queries!  Built for modern data teams and definitely fit for purpose!

If you want to read more about turning the traditional BI process on its head, read our full whitepaper, Cloud BI: Time to unlearn traditional BI development.

Check out the whitepaper