Data Warehouse Development Process
4.0 (747 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
10,907 students enrolled

Data Warehouse Development Process

Specific aspects of Data Warehouse Development Process
4.0 (747 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
10,907 students enrolled
Created by Sid Inf
Last updated 5/2019
English [Auto]
Current price: $34.99 Original price: $49.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 4 hours on-demand video
  • 1 downloadable resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Understand various stages in Data Warehouse development process
  • Various processes like Waterfall model, V model and Agile methods
  • Specific aspects of Data Warehouse development process
  • Importance of the various phases and the practicality of each phase
  • Overview of various issues and Project Management issues to be considered in the Data warehouse and Business Intelligence projects
  • Basics of Data Warehouse Concepts
  • Terms and terminologies used in a Data Warehouse and Business Intelligence projects

Data is the new asset for the enterprises. And, Data Warehouse store the data for better insights and knowledge using Business Intelligence. Development of an Enterprise Data Warehouse has more challenges compared to any other software projects because of the 

  • Challenges with data structures
  • The way data is evaluated for it's quality
  • Complex business rules/validations
  • Different development methods (various SDLC models like Water Fall model, V model, Agile Model, Incremental model, Iterative model)
  • Regulatory requirements for various domains like finance, telecom, insurance, Retail and IME
  • Compliance from third party governing bodies
  • Extracting data for various visualization purposes

In this course, we talk about the specific aspects of the Data Warehouse Development process taking real time practical situations and how to handle them better using best practices for sustainable, scalable and robust implementations.

Who this course is for:
  • Freshers/Engineering Graduates who are looking for placements
  • Software Engineers from different technology background who want to explore the Data Warehouse and Business Intelligence development process
  • Software Engineers who are already part of any Data Warehouse and Business Intelligence Projects
  • Project Mangers
  • Non IT professionals who like to learn how data is handled in enterprises
  • Technology experts and Team Leaders
  • Architects and Data Modelers
  • Data Scientists and Big Data Experts who want to understand the practical Data Warehouse Process
  • Database Administrators who want to explore the DWH/ETL/BI areas
  • Mainframe developers who want to switch their carrier into the Data Warehouse stream
Course content
Expand all 68 lectures 04:11:02
+ Introduction
2 lectures 07:37

We will start with the reason for building this course, the topics covered and who will benefit the most from this course. 

Preview 06:00

A quick note on the practical exercises, documents and the material. 

Practical Exercises, Documents and Material
+ Different Categories of Data Warehouse Implementations
1 lecture 04:59

In this lecture, we talk about the different categories of Data warehouse Projects and how those will be used in the course with real time examples. 

Different Categories of Data Warehouse Projects
+ Keywords/Terms used in this course
5 lectures 20:11

In this section, we define the terms we used in this project. 

  • Ad-Hoc Query
  • Business Intelligence
  • CRM
  • Database
  • DataMart
  • Data Mining
  • Data Visualization
  • Data Warehouse
  • Dimension Table
  • Fact Table
Data warehouse and Business Intelligence Glossary Part 1

This is part 2 of defining the terms and Keywords.

  • Granularity
  • Operational System
  • Partitioning
  • Replication
  • Reconciliation
  • MDM/Reference Data
Data warehouse and Business Intelligence Glossary Part 2

Additional information on the Data Warehouse and Business Intelligence glossary. 

Additional Information on all the related keywords for the DW/BI projects

Any requirement which specifies what the system should do is a functional requirement. 

-Business Rules
-Transaction corrections, adjustments and cancellations
-Authorization levels
-Audit Tracking
-External Interfaces
-Reporting Requirements
-Historical Data
-Legal or Regulatory  Requirements

What are Functional Requirements?

Any requirement which specifies how the system performs a certain function/activity is considered as Non Functional Requirement. 

  • Performance 
  • Scalability
  • Capacity
  • Availability
  • Reliability
  • Recoverability
  • Maintainability
  • Serviceability
  • Security
  • Regulatory
  • Manageability
  • Environmental
  • Data Integrity
  • Usability
  • Interoperability

What are Non Functional Requirements?
+ Methodologies used for Data Warehouse Software Development
2 lectures 08:12

In this lecture, we start with the basic understanding of what is a process in general for a physical product and the process for a software development.

Preview 04:39

Before we deep dive into the methodologies, lets start with the commonly used processes for an Enterprise Data Warehouse set up. 

What are the widely used methodologies for Enterprise Data Warehouse Set up?
+ Waterfall Model
11 lectures 43:07

The Waterfall Model was first Process Model to be introduced. It is also referred to as a linear-sequential life cycle model.  It is very simple to understand and use.  In a waterfall model, each phase must be completed fully before the next phase can begin.

What is the Waterfall Model and what are the different phases involved?

There are multiple versions of the stages which are defined for the Waterfall model. All of them follow the baseline structure of Requirements,Analysis, Development, Testing and Maintenance. 

In this course, we have used the below stages to split the various phases to understand it more better, 

  • System Requirements
  • Software Requirements
  • Analysis
  • Program Design
  • Coding
  • Testing
  • Operations
What are the different phases in Waterfall model?

This phase is focused on possible requirements of the system for the development are captured. Requirements are gathered subsequent to the end user consultation.

What is the Requirements phase?

Prior to beginning the actual coding, it's important to understand what actions are to be taken at what phase and what they should like.

What is the importance of the Analysis phase and how IT plays a part in this?

The requirement specifications are studied in detail in this phase and the design of the system is prepared. The design specifications are the base for the implementation and unit testing model phase.

What are the activities done during the Design phase?

Subsequent to receiving the system design documents (both technical and functional), the work is shared into various modules and the real coding is commenced. The system/application is developed into small coding units/modules and these units are later integrated in the subsequent phase. Every unit is tested for its functionality.

What are the activities done during the Coding phase?

The modules that are divided into units in the design phase are integrated into a complete system and tested for proper coordination among modules and system behaves as per the specifications. Once the testing is completed, the software product is delivered to the customer for UAT (User Acceptance Tests). And, once the tests are complete, the code will be deployed to production. 

What are the activities done during the Testing phase?

It is a never ending phase. Once the system is running in production environment, problems come up. The issues that are related to the system are solved only after deployment of the system. The problems arise from time to time and need to be solved; hence this phase is referred as maintenance.

And, any new regulatory requirements and new policies with in the organization also triggers more enhancements and changes to the application. 

What are the activities in Operations and Maintenance Phase?

Following the POC  (Proof of Concept) model for better results. This is not limited to only the waterfall model but for any implementation to know the complexity of the requirements and what we are getting into. 

Follow the POC (Proof of Concept) model for better results

Advantages of waterfall model:

  • Simple and easy to understand and use.
  • Easy to manage due to the rigidity of the model – each phase has specific deliverables and a review process.
  • Phases are processed and completed one at a time.
  • Works well for smaller projects where requirements are very well understood.
What are the advantages of using the Waterfall Model?

Disadvantages of waterfall model:

  • Once an application is in the testing stage, it is very difficult to go back and change something that was not well-thought out in the concept stage.
  • No working software is produced until late during the life cycle.
  • High amounts of risk and uncertainty.
  • Not a good model for complex and object-oriented projects.
  • Poor model for long and ongoing projects.
  • Not suitable for the projects where requirements are at a moderate to high risk of changing
What are the disadvantages of using the Waterfall Model?

Quiz on Water fall model

Quiz on Water fall model
3 questions
+ V Model
4 lectures 22:16

V- model is also called as the Verification and Validation model. Just like the waterfall model, the V-Shaped life cycle is a sequential path of execution of processes. Each phase must be completed before the next phase begins. Testing of the product is planned in parallel with a corresponding phase of development in V-model.

Following are the Verification phases in V-Model:

  • Business Requirement Analysis
  • System Design:
  • Architectural Design
  • Module Design
  • Coding
What are the different activities in the Verification phase?

Following are the Validation phases in V-Model:

  • Unit Testing
  • Integration Testing
  • System Testing
  • Acceptance Testing
What are the different activities in the Validation phase?

Advantages of V-model:

  • Simple and easy to use.
  • Testing activities like planning, test designing happens well before coding. This saves a lot of time. Hence higher chance of success over the waterfall model.
  • Proactive defect tracking – that is defects are found at early stage.
  • Avoids the downward flow of the defects.
  • Works well for small projects where requirements are easily understood.
What are the advantages of the V Model?

Disadvantages of V-model:

  • Very rigid and least flexible.
  • Software is developed during the implementation phase, so no early prototypes of the software are produced.
  • If any changes happen in midway, then the test documents along with requirement documents has to be updated.
What are the disadvantages of the V Model and When to opt for this model?

Quiz on 'V' model

Quiz on 'V' model
3 questions
+ Agile Model
6 lectures 25:55

An iterative life cycle model will not be started with a full specification of requirements. Instead, development begins by specifying and implementing just part of the software, which can then be reviewed in order to identify further requirements. This process is then repeated, producing a new version of the software for each cycle of the model.

Iterative Model

In incremental model the whole requirement is divided into various builds. Multiple development cycles take place here, making the life cycle a “multi-waterfall” cycle.  Cycles are divided up into smaller, more easily managed modules. Incremental model is a type of software development model like V-model, Agile model etc.

Incremental Model

Agile development model is a combination of both Iterative model and Incremental model. Software is developed in incremental, rapid cycles. This results in small incremental releases with each release building on previous functionality. Each release is thoroughly tested to ensure software quality is maintained. It is used for time critical applications.  

Agile Model

Advantages of Agile model:

  • Customer satisfaction by rapid, continuous delivery of useful software.
  • People and interactions are emphasized rather than process and tools. Customers, developers and testers constantly interact with each other.
  • Working software is delivered frequently (weeks rather than months).
  • Face-to-face conversation is the best form of communication.
  • Close, daily cooperation between business people and developers.
  • Continuous attention to technical excellence and good design.
  • Regular adaptation to changing circumstances.
  • Even late changes in requirements are welcomed
What are the advantages of Agile Model?

Disadvantages of Agile model:

  • In case of some software deliverable, especially the large ones, it is difficult to assess the effort required at the beginning of the software development life cycle.
  • There is lack of emphasis on necessary designing and documentation.
  • The project can easily get taken off track if the customer representative is not clear what final outcome that they want.
  • Only senior programmers are capable of taking the kind of decisions required during the development process. Hence it has no place for newbie programmers, unless combined with experienced resources.
What are the disadvantages of Agile Model?

Unlike the waterfall model in agile model very limited planning is required to get started with the project. Agile assumes that the end users’ needs are ever changing in a dynamic business and IT world. Changes can be discussed and features can be newly effected or removed based on feedback. This effectively gives the customer the finished system they want or need.

When to go for the Agile Model?

Quiz on 'Agile' model

Quiz on 'Agile' model
2 questions
+ Other SDLC Models
1 lecture 01:33

The Waterfall, V and the Agile model are not the only one's available for projects. These are the commonly used model for the Data warehouse implementations. There are other SDLC models which can be used for projects and those are. 

  • RAD model
  • Spiral model
  • Big Bang model
  • Prototype model
What are the other SDLC Models?

Quiz for Other SDLC models

Quiz for Other SDLC models
1 question
+ Which model to choose for the available models?
1 lecture 04:24

In this lecture, we discuss the real time scenarios which will help us decide on which model to choose for the Data Warehouse Development Process. 

Which is the best model?
+ Practical Implementation Steps
1 lecture 02:12

From this session onwards, we will start to see that practical implmemetaion steps for a Datawarehouse Development  project. Below are the various phases which we will discuss about. 

  • Project Planning
  • Requirements
  • Design
  • Development/Testing
  • Deploy
  • Operations and Maintenance
What are the practical implementation steps for the project?