Data Engineer - Commodities Technology
Industry-leading investment firm using innovative and cutting-edge technology, with data at the core of the investment process. They are seeking top-level data engineers to join their London office to focus specifically on data ingestion and transformation through web scrapes. You'll architect and build platforms integral to the sourcing, enriching and storing of the data that enables the investment process.
Partnering with investment professionals and data scientists, you will develop solutions that enable investment professionals to efficiently extract insights from data. This includes owing the ingestion (web scrapes, S3/FTP sync, sensor collection), transforming it into actionable insights (Spark, SQL, Kafka, Python/C++/R) and designing their interfaces (API, schema design, events).
The successful Data Engineer will have a passion for working with data and developing software to solve data processing challenges. Requirements:
- Experience building and managing large web scrapes
- Experience working with Google Cloud/AWS
- Strong Python skills
- Expert in some of the following: RDBMS, NoSQL, distributed compute platforms, such as Spark, Dask or Hadoop, Snowflake
- Minimum BS degree in Computer Science (or related); Master's or PhD degree would be ideal
- Experience with any of the following systems is a plus: Apache Airflow, AWS/GCE/Azure, Jupyter, Kafka, Docker, Nomad/Kubernetes
- Experience building ETL pipelines also a plus
- Hugely collaborative environment between teams, not siloed like other firms
- Competitive compensation: truly flat structure; feel valued for your input and be rewarded for great ideas
- Work with the latest technologies on complex problems
- Flexible working encouraged and regular socials
If you think you are a suitable candidate for the role and would like further info, please contact: Cajovi Daniel