Business Intelligence architecture
Let’s get started with bi architecture
You are viewing a minimalistic and standard illustration of BI architecture.
You may ask why minimalistic because we Can implement a staging area or data lake to make it more complex .
- First of all, the input data from various systems like OLTP systems, web URLs, files, etc. are extracted.
You may ask what is OLTP: OLTP Is an abbreviation for Online Transaction Processing that makes the wheels of the business turn.
in fact, the OLTP system can be handled the day-to-day transaction of an organization - Then the data is transformed and eventually loaded into a bigger database also known as a data warehouse. This process is called ETL or Extract, Transform, Load
- In the next step with the combination of the charts and analysis, we can drive the company into profit and proper decision-making.
What are the main differences between a Data lake, Data warehouse, and Staging Area?
Data Lake stores all data irrespective of the source and its structure, whereas Data Warehouse stores data in quantitative metrics with their attributes.
On the other hand, Data Lake uses the ELT(Extract Load Transform) process, while the Data Warehouse uses ETL(Extract Transform Load) process.
But now about the staging area:
In a Data Warehousing Architecture, a Data Staging Area is mostly necessary for time considerations. In the other words, before data can be incorporated into the Data Warehouse, all essential data must be readily available.
It is not possible to retrieve all data from all Operational databases at the same time because of varying Business Cycles, Data Processing Cycles, Hardware, and Network Resource Restrictions, and Geographical Variables.
For example, It’s reasonable to extract sales data on a daily basis, but daily extracts aren’t suitable for financial data that needs to be adapted at the end of the month.
extracting “customer” data from a database in Singapore at noon eastern standard time may be appropriate, but it is not appropriate for “customer” data in a Chicago database.
We will also cover the difference in the importance of having a data warehouse in traditional bi, unlike self-service bi which can work without a data warehouse in the next articles.
I have been working as a senior Business Intelligence Developer such as (Microsoft BI Services, QlikView, Qlik sense, and Power BI) since 2007, what’s more, I have been teaching BI courses such as (QlikView, Qlik Sense, Power BI, and SSRS) for 8 Years.
I want to address that I have been a consultant in various companies such as (Pharmaceutical corporates, port and maritime, insurance companies etc.).
I am capable of covering tasks such as (data cleansing, Designing dashboards, and KPI identification, and also I am well experienced in mentioned fields.
I have studied software engineering for 4 years bachelor’s degree In addition , I have published two books about business intelligence (QlikView with business intelligence approach, organizational key performance indicator)
I have passed the MCT and MCSA courses and I have been qualified for them.
Leave a Reply
Want to join the discussion?Feel free to contribute!