Big Data/Event Streaming Engineer - Experience


Curitiba, PR, BR

Company:  ExxonMobil

Job Role Summary

Work with business and technical teams to provide technical support and development expertise for overall big data pipeline using Nifi and Kafka, Hadoop Spark workflows, Hadoop Tables and implement data models in Data Science & Analytics portfolio.

Job Role Responsibilities

• Develop or enhance Nifi, Kafka and Spark flows for processing of structured and unstructured data.
• Develop or enhance Hadoop workflows including data ingestion, extraction and processing of structured and unstructured data.
• Engage in proof of concepts, technical demos and interaction with customers and other technical teams
• Proactively automate monitoring, alerting based on workflow needs
• Work to utilize common CI/CD best practices across different teams (data scientists, data visualization analysts, web developers etc.) when working on the same project
• Ensure solution compliance to technical design standards and architecture
• Explore and develop skills in newer technologies according to big data technology roadmap

Expected Level of Proficiency

• Minimum Bachelor's degree from a recognized university in Computer Science or equivalent; minimum GPA - 6.5 (out of 10.0)
• 3-5 Years of experience working with Hadoop
• Applicants must have the following skills:
o Applied Experience with Spark
o Applied Experience with Kafka
o Applied Experience with Scala and/or Python
o Applied Experience with NiFi
o Applied Experience with Solr
o Experience with Oozie
o Experience with the Cloudera Data Platform (CDP)
o Experience with Flink-SQL
o Familiarity with Azure DevOps (nice to have, but not required)
o Familiarity with SQL (nice to have, but not required)
o Familiarity using Git (nice to have, but not required)
o Experience in UNIX or UNIX-like systems
o Experience with SCRUM or other similar Agile framework
• Soft Skills:
o Strong verbal and written communication skills
o Self-starter and independent worker
o Strong problem resolution skills
o Ability to work with data scientists and develop seamless workflows
o Ability to interface with customers, technology partners, testing, architecture and analysis groups
o  Applicants will be given preference for having the following skills:



Job Segment: Database, Engineer, Computer Science, Unix, SQL, Technology, Engineering