Senior Analyst - Data Engineer

Main Purpose:

▪Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake.
▪
Maintain the integrity and quality of the data lake, enabling accurate and actionable insights for data scientists and informed decision-making for business stakeholders.
▪Utilize extensive knowledge of data engineering and cloud technologies to enhance the organization's data infrastructure, promoting a culture of data-driven decision-making.
▪
Apply data engineering expertise to define and optimize data pipelines using advanced concepts to improve the efficiency and accessibility of data storage.
▪Own the development of an extensive data catalog, ensuring robust data governance and facilitating effective data access and utilization across the organization.

Knowledge Skills and Abilities, Key Responsibilities:

Key Responsibilities

•Contribute to the development of scalable and performant data pipelines on Databricks, leveraging Delta Lake, Delta Live Tables (DLT), and other core Databricks components.

•Experience with CI/CD and DevOps practices specifically tailored for the Databricks environment.

•Monitor and optimize the cost-efficiency of data operations on Databricks, ensuring optimal resource utilization.

•Utilize a range of Databricks tools, including the Databricks CLI and REST API, alongside Apache Spark™, to develop, manage, and optimize data engineering solutions.

Key Relationships and Department Overview:

Key Relationships
•Internal – Data Engineering Manager
•Developers across various departments, Managers of Departments in other regional hubs of Puma Energy
•External – Platform providers