Python Data Engineer

Scrape public data, build pipelines (bronze/silver/gold), and create APIs. Bring in industry best-practices around creating and maintaining robust data pipelines for complex data projects with / without AI component: programmatically retrieve (unstructured mostly) data from several static and real-time sources (incl. web scraping, API use). Structure this data into a structured format o Harmonize the data, into a common format and store it in a dedicated database. Schedule the different jobs into a dedicated pipelinerendering results through dynamic interfaces incl. web / mobile / dashboard with ability to log usage and granular user feedbacksperformance tuning and optimal implementation of complex Python scripts, SQL,…Industrialize ML / DL solutions and deploy and manage production services; proactively handle data issues arising on live appsPerform ETL on large and complex datasets for AI applications - work closely with data scientists on performance optimization of large-scale ML/DL model finetuningBuild data tools to facilitate fast data cleaning and statistical analysisBuild and ensure data architecture is secure and compliantResolve issues escalated from Business and Functional areas on data quality, accuracy, and availabilityWork closely with APAC IT Transformation and coordinate with a fully decentralized team across different locations in APAC and global HQ (Paris).

Expert in structured and unstructured data in traditional and Big data environments Oracle / SQLserver, MongoDB, Hive / Pig, BigQuery and Spark. Have excellent knowledge of Python programming both in traditional and distributed models (PySpark). Expert in shell scripting and writing schedulers. Hands-on experience with Cloud - deploying complex data solutions in hybrid cloud / on-premise environment both for data extraction / storage and computation. Well versed with DevOps best practices like containerization, CICD pipeline (Jenkins and Maven). Hands-on experience in deploying production apps using large volumes of data with state-of-the-art technologies like Dockers, Kubernetes and Kafka. Strong knowledge of data security best practices.