Data Modernisation Azure
& Databricks​​

A North American investment banking firm faced a challenge with its obsolete Hadoop system hardware used for data and analytics, creating a need for system modernisation.

The bank is facing regulatory pressure given the lack of accuracy and quality in the reports provided.

Increasing costs given the amount of manual work and reconciliation processes required.

• Developed a future state architecture blueprint for the migration
• Set up the new platform on Databricks and helped migrate existing Cloudera workflows to Azure Databricks workflows
• Created 700+ new data ingestion pipelines to bring data from on-premise operational database and external vendor files to the newly created Analytics Hub Data Lake
• The Data Lake was integrated with Power BI, Tableau, SAS and Dataiku
• Over 50+ TB of historical data was migrated from Hadoop HDFS to the Data Lake

• 35 Applications migrated in a record time of 2 years​
• 5 New applications migrated with a pipeline and budget for 20 new applications​
• Modern platform which reduced licensing costs for the client while lowering the overall TCO