Big Data Specialist

Our client in the Banking industry is seeking a Big Data Specialist that will join their team in Jhb.

Responsibilities will include:

  • Data munging, visualization, interpreting and effectively communicating data findings
  • Assuring the quality and management of scraped data
  • Extrapolating data using advanced computer modelling, triaging code problems, and pruning
  • Develop, construct, test and maintain highly scalable data management systems including processing data using
  • Collect and store data, do real-time or batch processing, and serve it for analysis via an API
  • Log and handle errors, identify when to scale up, ensure seamless integration with new data management technologies and build human – fault-tolerant pipelines
  • Design web services for swift data tracking and query data at high speeds
  • Test software prototypes, propose standards, and smoothly transfer them to operations

Required Skills:

  • Java
  • Hadoop
  • Hive
  • Zeppelin
  • SQL
  • Apache Camel
  • Apache Flink
  • Apache Kafka
  • Docker
  • Kubernetes
  • OpenShift
  • Spark
  • Streaming Analytics Manager
  • EIP (Enterprise Integration packages)
  • REST API
  • DevOps