25th July 2022
Data Engineer
Big Data
London
£73,000
Data Engineer
London
£73,000
Spencer Rose are looking for a Data Engineer top help adopt a Cloud-first approach, delivering an advantage to how Data is looked at. The Data Engineer will work closely with all areas of the Business and technology teams. The Data Engineer will be involved in the migration to a cloud-based architecture that can scale performance on demand.
The Data Engineer’s responsibilities:
- Translate raw data into usable datasets to support business requirements.
- Deliver robust, high-performing and scalable solutions for loading, transforming and processing company data.
- Develop adhering to the current development best practices whilst contributing to the continual improvement of these practices.
- Maintain formal data and metadata structures (lineage) to support integrity and searchability of information via a data dictionary.
- Working collaboratively with:
Principal Data Engineer and Data engineering Lead for guidance and review of tasks
Our Data Architect to define and build data marts
Our Data Platform team to ensure high level of performance and provide support for delivered solutions
- Provide consultation to Project Managers, Business Analysts and Managers around current processes, best practices and ideas to solve business challenges
The Data Engineer should have the following Skills:
- Experience of end to end implementation of Data Engineering and Analytics over a Cloud Infrastructure – Preferably with Azure
- Technical versatility with Azure offerings including Databricks or Spark, Azure Data Factory, Azure Data Lake, DevOps
- Strong knowledge of Big Data technologies such as Nifi, Hive, Spark, Kafka HBase
- Excellent troubleshooting, performance tuning and security awareness
- Experience building or maintaining ETL processes and developing data pipelines using agile & DevOps
- Have experience with using CI/CD tooling to analyse, build, test and deploy your code.
Consultant - Chloe Aboud
Telephone: 0207 392 7516
Email: chloe.aboud@spencer-rose.com
Share This Position
