Data Engineer - 8 months - Cambridgeshire - DV Cleared - £700
Data Engineer required for an 8 month contract based in Cambridgeshire. Our client is looking for someone to support an Agile development project which is building bespoke algorithms and applications.
The successful candidate will design, build and integrate data from various resources to enable a team of data scientists to rapidly leverage those data sets.
*Candidates are required to hold VALID DV Clearance to be considered.*
- Create, manage & monitor data flows with Apache NiFi.
- Extend Apache Nifi using scripts such as Groovy and Python.
- Be responsible for Hadoop data loading.
- Extensive Apache NiFi experience with the ability to write custom processors.
- Knowledge of Apache NiFi administration and security.
- Experience with Web scraping including the use of REST APIs, PKI authentication & interaction with SharePoint and RSS Feeds.
- Python skills including experience of GDAL, NumPy and Scipy.
- Apache Kafka usage, configuration and optimisation.
- Knowledge of ElasticSearch data loading & optimisation
- PostgreSQL & PostGIS usage, configuration and administration
- Experience with large scale geospatial data handling.
- Prior experience working with satellite imagery formats