Data Engineer

As a Senior Data Engineer, you will be responsible for designing, developing and optimizing complex data pipelines, supporting data modeling, and contributing to the architecture that supports big data processing and analytics to cutting-edge cloud solutions that drive business growth. You will lead the design and implementation of scalable, high-performance data solutions on AWS and mentor junior team members. This role demands a deep understanding of AWS services, big data tools, and complex architectures to support large-scale data processing and advanced analytics. Key Responsibilities:Design and develop robust, scalable data pipelines using AWS services, Python, PySpark, and SQL that integrate seamlessly with the broader data and product ecosystem.Lead the migration of legacy data warehouses and data marts to AWS cloud-based data lake and data warehouse solutions.Optimize data processing and storage for performance and cost.Implement data security and compliance best practices, in collaboration with the IT security team.Build flexible and scalable systems to handle the growing demands of real-time analytics and big data processing.Work closely with data scientists and analysts to support their data needs and assist in building complex queries and data analysis pipelines.Collaborate with cross-functional teams to understand their data needs and translate them into technical requirements.Continuously evaluate new technologies and AWS services to enhance data capabilities and performance.Create and maintain comprehensive documentation of data pipelines, architectures, and workflows.Participate in code reviews and ensure that all solutions are aligned to pre-defined architectural specifications.Present findings to executive leadership and recommend data-driven strategies for business growth.Communicate effectively with different levels of management to gather use cases/requirements and provide designs that cater to those stakeholders.Handle clients in multiple industries at the same time, balancing their unique needs.Provide mentoring and guidance to junior data engineers and team members.

3+ years of experience in a data engineering role with a strong focus on AWS, Python, PySpark, Hive, and SQL. Proven experience in designing and delivering large-scale data warehousing and data processing solutions. Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.