Loading...
 

Big Data Engineer (Apache Kafka / Nifi)

Location: 

Curitiba, PR, BR

Company:  ExxonMobil

Who We are

We are more than just one of the largest energy companies. ExxonMobil is the start of what could be an incredible journey. It’s the place where people think independently, take initiative, grow to their full potential and shape the future of energy. We strongly believe that diversity drives innovation so our inclusive workplaces help everyone thrive, regardless of their background. Safety and wellbeing of our employees is at the heart of everything we do. Our skills and competencies are built through training and work experiences in a wide range of assignments. With the use of cutting edge technology ExxonMobil holds an industry leading inventory of resources to help meet our societies growing energy demands.

What you will do

  • Develop or enhance Big Data pipelines using Nifi, Kafka, Spark and/or Flink-SQL for processing of structured and unstructured data
  • Proactively automate monitoring, alerting based on workflow needs
  • Develop or enhance Hadoop workflows including data ingestion, extraction and processing of structured and unstructured data
  • Engage in proof of concepts, technical demos and interaction with customers and other technical teams
  • Work to utilize common CI/CD best practices across different teams (data scientists, data visualization analysts, web developers etc.) when working on the same project
  • Explore and develop skills in newer technologies according to big data technology roadmap

What you need to have

  • Working experience with Big Data Event streaming Technologies
  • Applied Experience with Kafka and/or Nifi
  • Applied Experience with Scala and/or Python
  • Familiarity with the SQL language
  • Cloud experience (Azure and/or AWS)

 

Nice to have skill set:

  • Experience with Spark and / or Flink-SQL
  • Experience with the Cloudera platform or products (e.g.: CDP, Nifi, Kafka, Minifi)
  • Familiarity with Azure DevOps
  • Familiarity using Git
  • Experience in UNIX or UNIX-like systems
  • Experience with SCRUM or other similar Agile framework


Job Segment: Database, Engineer, SQL, Unix, Technology, Engineering