Senior Data Engineer Python, Kafka, StreamSet, Airflow, Postgres, Docker Our client, a global telecommunications firm based in the City of London seeks a senior data engineer to work with large complicated datasets with the goal of transforming the data pipeline providing data downstream for the data science teams. You will be responsible for: * The development, maintenance, and deployment of data integration and ETL pipelines * Setting up rules, alerting, and monitoring for data consistency and quality * Converting and migrating current (legacy) data pipelines * Working together with the data scientists and operations to help them provide the right data at the right time to the business, our partners, value added resellers, and customers * Automating business and operations reports and alerts What we need from you: * Managing distributed systems and clusters * Deployment, provisioning, and automation tools. (e.g. git, Jenkins, Sonar, RPM) * Linux system administration * Programming in scripting languages, e.g. bash, python * Building (complex) ETL pipelines If this role sounds of interest or more information, please apply and you will be contacted.
