We are taking a top-down approach to data architecture. In this lecture we will sketch the data architecture blueprint that in the remainder of the course we will deconstruct and detail component-by-component.

Be sure to download the architecture as a PDF in the sources list!

How do we get data into our data ecosystem? We introduce what is called the data onboarding blueprint and what mechanisms come into play when getting in.

Be sure to download the example data onboarding document in the sources list.

Taking data into the raw storage, enrich and potentially analyze them on-the-fly is what we look at in this lecture.

What are micro services and how do we utilize them? We also talk about integrated storage and how we can potentially already write to it when ingesting data through the on-the-fly enrichments.

The core beating heart of our data architecture blueprint is the integrated ETL, taking raw data and moving into an as broad as possible integrated schema and storage. We'll explain how we do this and what we need to think of when doing this.

We cannot tackle everything with just the integrated ETL. In this lecture we will look at case-specific ETL and storage as we move further down the funnel of going from raw data to tangible data products.

Just having data processed and available doesn't mean that people can use it. How can we serve the data out such that stakeholders and business can reap what we sow.

Data scientists need to have untethered access to our data to maximize the modelling and analyses they perform. How can we ensure this without jeopardizing the operational constructs of our architecture? Find out in this lecture.

You've probably heard the term: data mesh. What is it? How does it fit in with our data architecture blueprint? Is it the next evolution or just something that will come and go? What about organizational changes required to shift towards data mesh? We answer these questions in this lecture.

Let's cycle back to the data architecture blueprint now that we have extensively talked about the different components that comprise it.

In this lecture we look at how to realize a data architecture for a large consumer brand that required to analyze all the logs generated by their consumer products, IT ecosystem and corporate infrastructure. What components of our blueprint did we use and how did we combine them?

In this use case we look at applying the data architecture blueprint to consolidate across spuriously-connected IoT sensor data. Specific applications that monitor uptime, predict maintenance and analyze anomalies were implemented as case-specific components. Find out how.

A holistic view on a large media brand that analyzes the behavior of visitors and customers in an omnichannel setting where online and offline information is combined into streaming insights.

One of the core software products that my own company, UnderstandLing, built is an HR and recruitment solution that uses Natural Language Processing to help candidates find jobs they love and recruiters find the most driven candidate for their vacancies. We'll see how the data architecture blueprint can even be applied to settings where ad-hoc and ill-defined data analyses are not the most crucial components but an operational application is.

The culmination of everything we have seen and can do with data architectures: an abstraction of the blueprint that will allow us to just instantiate any data architecture by configuring the components' orchestration and logic to be executed. This lecture talks about another product built at UnderstandLing called CEMistry that does exactly this. Join me in finding out how exactly we managed to make this happen.

Now that we have seen and talked about the data architecture blueprint and looked at how we can apply it to vastly different use cases, we wrap up the course and talk about next steps to make in your data (architecture) career.