IT_IT New Technologies_1 (100)

• Design, develop, and maintain scalable data pipelines and systems.
• Work closely with data scientists and analysts to ensure data quality and availability.
• Implement data integration and transformation processes using PySpark and Python.
• Optimize and maintain SQL databases and queries.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipeline issues to ensure data integrity and performance.

Required Skills and Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in data engineering.
• Proficiency in PySpark, Python, and SQL.
• Experience with Azure Databricks is a plus.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork abilities.

Responsibilities

• Design, develop, and maintain scalable data pipelines and systems.
• Work closely with data scientists and analysts to ensure data quality and availability.
• Implement data integration and transformation processes using PySpark and Python.
• Optimize and maintain SQL databases and queries.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipeline issues to ensure data integrity and performance.

Qualifications

Preferred Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in data engineering.
• Proficiency in PySpark, Python, and SQL.
• Experience with cloud platforms such as Azure, AWS, or Google Cloud.
• Knowledge of data warehousing concepts and technologies.
• Familiarity with ETL tools and processes.