ETL - Telecom - Garage Education
]اﻟﻤﺼﺪر :ﻣﺠﻤﻮﻋﺔ اﻟﻔﯿﺪﻳﻮھﺎت اﻟﻤﻨﺸﻮرة ﻋﻠﻰ ﻗﻨﺎة Garage Educationﺑﻮاﺳﻄﺔ اﻟﻤﮫﻨﺪس /ﻣﺼﻄﻔﻰ ﻋﻼء،
وﺑﮫﺎ ﺗﻢ اﻟﺘﻄﺒﯿﻖ ﻣﻦ ﺧﻼل ﻟﻐﺔ ،Scalaاﻟﮫﺪف ﻣﻦ ھﺬا اﻟﻤﺴﺘﻨﺪ ھﻮ ﺗﻜﺮار ﻓﻜﺮة اﻟﺴﻠﺴﻠﺔ وﻟﻜﻦ ﺑﺎﺳﺘﺨﺪام
- [SQL Server Integration Services SSISﺳﻜﺮﻳﺒﺖ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت
إﺣﺪى ﺷﺮﻛﺎت اﻻﺗﺼﺎﻻت ﺗﻮاﺻﻠﺖ ﻣﻌﻚ ﺑﺼﻔﺘﻚ ETL Developer/Consultantﻟﻠﻘﯿﺎم ﺑﻌﻤﻠﯿﺔ ﺗﺨﺰﻳﻦ اﻟﺒﯿﺎﻧﺎت
وﻓﻘﺎً ﻟﻤﺎ ﻳﻠﻲ
.1اﻟﺸﺮﻛﺔ ﻟﺪﻳﮫﺎ ﺳﯿﺴﺘﻢ ﻳﻘﻮم ﺑﺤﻔﻆ ﻣﻠﻒ csvﺑﺸﻜﻞ دوري ﻛﻞ 5دﻗﺎﺋﻖ ،ھﺬا اﻟﻤﻠﻒ ﻳﺸﺘﻤﻞ ﻋﻠﻰ
اﻟﺒﯿﺎﻧﺎت اﻷﺳﺎﺳﯿﺔ ﻟﻠﺤﺮﻛﺎت اﻟﻤﺨﺘﻠﻔﺔ اﻟﺘﻲ ﻗﺎم ﺑﮫﺎ اﻟﻌﻤﻼء ﺧﻼل ﻓﺘﺮة زﻣﻨﯿﺔ ﻣﺤﺪدة.
.2ﻗﺪﻣﺖ اﻟﺸﺮﻛﺔ ﻟﻚ اﻟﺠﺪول اﻟﺘﺎﻟﻲ واﻟﺬي ﻳﻮﺿﺢ اﻟﺒﯿﺎﻧﺎت اﻟﻤﺨﺰﻧﺔ ﻓﻲ ﻣﻠﻒ csvو
Column Data Length Is Sample
Name Type Nullable
ID Int False 156
IMSI String 9 False 310120265
IMEI String 14 True 490154203237518
CELL Int False 123
LAC Int False 12
EVENT_TYPE String 1 True 1
EVENT_TS Timestamp False 16/1/2020 7:45:43
اﻟﻤﻌﺎﻟﺠﺔ اﻟﻤﻄﻠﻮﺑﺔ إﺗﻤﺎﻣﮫﺎ ﻋﻠﻰ ھﺬه اﻟﺒﯿﺎﻧﺎت ﻗﺒﻞ ﺗﺨﺰﻳﻨﮫﺎ ﻓﻲ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت.3
Column Mapping Rules Target
Name Model
ID As-is Transaction_id
IMSI As-is, r eject the record if null IMSI
IMSI Join with IMSI reference and get subscriber id, subscriber_id
add -99999 if null
IMEI Substring first 8 chars, i f null or size is less TAC
than 15 add -99999
IMEI Substring last 6 chars, if null or size is less SNR
than 15 add -99999
IMEI As-is IMEI
CELL As-is, r eject the record if null CELL
LAC As-is, r eject the record if null LAC
EVENT_TYPE As-is EVENT_TYPE
EVENT_TS Validate the timestamp format to be EVENT_TS
YYYY-MM-DD HH:MM:SS, reject the record if
null
ﻣﻊ ﺗﺴﺠﯿﻞ اﺳﻢ اﻟﻤﻠﻒ، ﺳﯿﺘﻢ ﺗﺨﺰﻳﻨﮫﺎ ﻓﻲ ﺟﺪول ﻣﺴﺘﻘﻞ،rejected records اﻟﺤﺮﻛﺎت اﻟﻤﺮﻓﻮﺿﺔ.4
csv اﻷﺻﻠﻲ
ھﻨﺎك ﺑﻌﺾ اﻟﺒﯿﺎﻧﺎت اﻹﺿﺎﻓﯿﺔ اﻟﻤﻄﻠﻮب ﺗﺴﺠﯿﻠﮫﺎ، ﺧﻼل ﻋﻤﻠﯿﺔ ﺗﺴﺠﯿﻞ اﻟﺒﯿﺎﻧﺎت ﻓﻲ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت.5
ﻟﻀﻤﺎن ﺟﻮدة ﻋﻤﻠﯿﺔ ﺗﺨﺰﻳﻦ اﻟﺒﯿﺎﻧﺎت
csv ﻋﺪد اﻟﺤﺮﻛﺎت ﻓﻲ ﻣﻠﻒ.a
ﻋﺪد اﻟﺤﺮﻛﺎت اﻟﺘﻲ ﺗﻢ ﺗﺨﺰﻳﻨﮫﺎ ﻓﻲ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت.b
ﻋﺪد اﻟﺤﺮﻛﺎت اﻟﻤﺮﻓﻮﺿﺔ ﻟﻌﺪم ﺗﺤﻘﻖ اﻟﺸﺮوط اﻟﻤﻄﻠﻮﺑﺔ.c
csv رﺑﻂ اﻟﺒﯿﺎﻧﺎت اﻟﻤﺨﺰﻧﺔ ﻓﻲ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت ﻣﻊ اﻟﻤﻠﻒ اﻷﺻﻠﻲ.d
اﻟﻰcsv اﻧﻘﻞ ﻣﻠﻒ، ﺑﻌﺪ إﺗﻤﺎم ﻋﻤﻠﯿﺔ ﺗﺨﺰﻳﻦ اﻟﺒﯿﺎﻧﺎت ﻓﻲ ﻗﺎﻋﺪة اﻟﺒﯿﺎﻧﺎت وﻓﻘﺎ ﻟﻠﺸﺮوط اﻟﺴﺎﺑﻘﺔ.6
ﻓﻮﻟﺪر آﺧﺮ
