Data is the new asset for the enterprises. And, Data Warehouse store the data for better insights and knowledge using Business Intelligence. Development of an Enterprise Data Warehouse has more challenges compared to any other software projects because of the
In this course, we talk about the specific aspects of the Data Warehouse Development process taking real time practical situations and how to handle them better using best practices for sustainable, scalable and robust implementations.
In this lecture, we talk about the different categories of Data warehouse Projects and how those will be used in the course with real time examples.
In this section, we define the terms we used in this project.
This is part 2 of defining the terms and Keywords.
Additional information on the Data Warehouse and Business Intelligence glossary.
Any requirement which specifies what the system should do is a functional requirement.
-Transaction corrections, adjustments and cancellations
-Legal or Regulatory Requirements
Any requirement which specifies how the system performs a certain function/activity is considered as Non Functional Requirement.
In this lecture, we start with the basic understanding of what is a process in general for a physical product and the process for a software development.
Before we deep dive into the methodologies, lets start with the commonly used processes for an Enterprise Data Warehouse set up.
The Waterfall Model was first Process Model to be introduced. It is also referred to as a linear-sequential life cycle model. It is very simple to understand and use. In a waterfall model, each phase must be completed fully before the next phase can begin.
There are multiple versions of the stages which are defined for the Waterfall model. All of them follow the baseline structure of Requirements,Analysis, Development, Testing and Maintenance.
In this course, we have used the below stages to split the various phases to understand it more better,
This phase is focused on possible requirements of the system for the development are captured. Requirements are gathered subsequent to the end user consultation.
Prior to beginning the actual coding, it's important to understand what actions are to be taken at what phase and what they should like.
The requirement specifications are studied in detail in this phase and the design of the system is prepared. The design specifications are the base for the implementation and unit testing model phase.
Subsequent to receiving the system design documents (both technical and functional), the work is shared into various modules and the real coding is commenced. The system/application is developed into small coding units/modules and these units are later integrated in the subsequent phase. Every unit is tested for its functionality.
The modules that are divided into units in the design phase are integrated into a complete system and tested for proper coordination among modules and system behaves as per the specifications. Once the testing is completed, the software product is delivered to the customer for UAT (User Acceptance Tests). And, once the tests are complete, the code will be deployed to production.
It is a never ending phase. Once the system is running in production environment, problems come up. The issues that are related to the system are solved only after deployment of the system. The problems arise from time to time and need to be solved; hence this phase is referred as maintenance.
And, any new regulatory requirements and new policies with in the organization also triggers more enhancements and changes to the application.
Following the POC (Proof of Concept) model for better results. This is not limited to only the waterfall model but for any implementation to know the complexity of the requirements and what we are getting into.
Advantages of waterfall model:
Disadvantages of waterfall model:
Quiz on Water fall model
V- model is also called as the Verification and Validation model. Just like the waterfall model, the V-Shaped life cycle is a sequential path of execution of processes. Each phase must be completed before the next phase begins. Testing of the product is planned in parallel with a corresponding phase of development in V-model.
Following are the Verification phases in V-Model:
Following are the Validation phases in V-Model:
Advantages of V-model:
Disadvantages of V-model:
Quiz on 'V' model
An iterative life cycle model will not be started with a full specification of requirements. Instead, development begins by specifying and implementing just part of the software, which can then be reviewed in order to identify further requirements. This process is then repeated, producing a new version of the software for each cycle of the model.
In incremental model the whole requirement is divided into various builds. Multiple development cycles take place here, making the life cycle a “multi-waterfall” cycle. Cycles are divided up into smaller, more easily managed modules. Incremental model is a type of software development model like V-model, Agile model etc.
Agile development model is a combination of both Iterative model and Incremental model. Software is developed in incremental, rapid cycles. This results in small incremental releases with each release building on previous functionality. Each release is thoroughly tested to ensure software quality is maintained. It is used for time critical applications.
Advantages of Agile model:
Disadvantages of Agile model:
Unlike the waterfall model in agile model very limited planning is required to get started with the project. Agile assumes that the end users’ needs are ever changing in a dynamic business and IT world. Changes can be discussed and features can be newly effected or removed based on feedback. This effectively gives the customer the finished system they want or need.
Quiz on 'Agile' model
The Waterfall, V and the Agile model are not the only one's available for projects. These are the commonly used model for the Data warehouse implementations. There are other SDLC models which can be used for projects and those are.
Quiz for Other SDLC models
In this lecture, we discuss the real time scenarios which will help us decide on which model to choose for the Data Warehouse Development Process.
From this session onwards, we will start to see that practical implmemetaion steps for a Datawarehouse Development project. Below are the various phases which we will discuss about.
Business Intelligence Consultant and Trainer with 14+ years of extensive work experience on various client engagements. Delivered many large data warehousing projects and trained numerous professionals on business intelligence technologies. Extensively worked on all facets of data warehousing including requirement gathering, gap analysis, database design, data integration, data modeling, enterprise reporting, data analytics, data quality, data visualization, OLAP.
Has worked on broad range of business verticals and hold exceptional expertise on various ETL tools like Informatica Powercenter, SSIS, ODI and IDQ, Data Virtualization, DVO, MDM.