Senior Azure Data Engineer

PAKISTAN

HIRING NOW!

A Senior Azure Data Engineer is required, who possesses over 10 years of continuous programming experience and a minimum of 5 years of continuous experience with Microsoft Azure.

Please note, this is a senior role with the expectation that this individual will be able to work independently, query requirements, prepare documentation and tests, as well as undertake the delivery of programming code, which is robust, scalable and reusable. A strong command of spoken and written English is essential.

Duties and responsibilities:

Highly experienced in Azure cloud services.
Highly experienced in designing enterprise data warehouse.
Highly experienced as an ETL/ELT Developer using various ETL/ELT tools such as Azure Synapse Pipelines & Azure Data Factory, Apache Spark Pools using python scripts.
Highly experienced in Azure SQL databases & SQL Programming.
Highly experienced in building end-end scalable data solution, from sourcing raw data, transforming data to producing analytics reports.
Highly experienced in building high performance, scalable, robust production processes with quality & security controls.
Experience in Azure platform services such as blob storage, event hubs, Azure monitoring services.
Experience in creating data structures optimized for storage and various query patterns for e.g. Parquet.
Experience in building secured Power BI reports, dashboard, paginated reports.
Experience in working in an Agile SDLC methodology.
Experience in Azure DevOps Services using Azure Git Repos, deployment artefacts and release packages for test & production environment.
Experienced liaising with business users to capture & convert business requirements into technical requirements.
Knowledge in financial data will be an added advantage.

Technical Skills

Strong knowledge in modern data warehouse design, build and test approaches.
Understand the concepts and principles of data modelling across Data lakes, data partition, 3NF, dimensional modelling.
Strong knowledge across database querying languages using SQL, Python, Scala for writing code to query, transform business logic, create and manage database objects.
Proficient in Azure Synapse Analytics to build, debug, schedule and monitor pipelines & jobs and ingest both structured and unstructured data sources through flat files, SQL Store and API services.
Deployment of Azure resource through ARM templates.
Knowledge of Azure CLI using either Bash & Powershell scripting languages.
Azure Data Engineer certification will be an added advantage.

Required to

Contribute technically into delivering data improvements to an Azure enterprise data warehouse.
Translate business requirements into technical requirements.
Build data pipelines using Azure Synapse Analytics, Blob Storage, SQL DB, Apache Spark & other Azure services as required.
Ensure the accuracy of code quality (data integrity, validation, logging, exception management, Jobs run status along job details etc) as well as performance through best practices & Microsoft architectural guidelines.
Build reusable code, libraries and consumable frameworks for data sourcing, staging, transform, conforming and extraction.
Build, test and deploy production ETL/ELT data pipelines and associated software components.
Build Power BI reports, publish to Power BI workspaces & Apps & secure through AAD integration.
Write technical documentation to support the maintenance of production release.