Hadoop Operation Lead Engineer

Short Description

Apple is hiring a Hadoop Operation Lead Engineer who has experience with Configuration Management and automation along with an Organized, focused on building, improving, resolving and delivering.

Job Description

Key Qualifications
  • Hands-on experience with handling production clusters (Hadoop, Spark) as well as strong development/automation skills and being very comfortable with reading and writing Python and Java codes
  • Overall 10+ years with at least 5+ years of Hadoop experience in large scale, multi-tenant production clusters (1000+ nodes with Queues) with a tools-first mindset 
  • You build tools for yourself and others to increase efficiency and to make hard or repetitive tasks easy and quick.'
  • Experience with Configuration Management and automation along with an Organized, focused on building, improving, resolving and delivering.
  • Good communicator in and across teams along with strong team management and leadership skills

Description
The role would primarily involve- 
  • Taking responsibility for maintaining and scaling large scale (1000+ nodes) production Hadoop and Spark clusters as well as managing a team of Hadoop Engineers providing mentorship and allocation of tasks along with technical help required by the engineering team
  • Strong knowledge of Kerberos for operation and debugging Kerberos issues in Hadoop clusters along with being the sole point of contact for Hadoop related critical issues coming from Application teams and internal clusters 
  • Being responsible for the implementation and ongoing administration of Hadoop infrastructure including monitoring, tuning, and troubleshooting and providing hardware architectural mentorship, plan and estimate cluster capacity, and craft roadmaps for the Hadoop cluster deployment
  • Improving scalability, service reliability, capacity, and performance of the cluster and applications running in the cluster
  • Triaging production issues when they occur with other operational teams, conducting ongoing maintenance across our large scale deployments across the world and writing automation code for running large BigData clusters
  • Working with development and QA teams to design Ingestion Pipelines, Integration APIs, and provide Hadoop ecosystem services and participating in on-call rotation supporting the Hadoop Ecosystem
  • Detailed experience in solving incidents, formulate theories and test hypothesis and narrow down possibilities to find the root cause

Education & Experience
Bachelors or Master Degree in Computer Science or similar discipline is required

Additional Requirements
  • Familiarity with geospatial data would be an added advantage
  • Apple is an Equal Opportunity Employer that is committed to inclusion and diversity. We also take affirmative action to offer employment and advancement opportunities to all applicants, including minorities, women, protected veterans, and individuals with disabilities.

Hadoop Operation Lead Engineer
Mid-Senior-level Technology | Information | Information Technology Full-time Information Technology | Engineering | Other Software Engineer | Hadoop | Data Engineer
Apple Inc. is an American multinational technology company headquartered in Cupertino, California that designs, develops, and sells consumer electronics, computer software, and online services. The company's hardware products include the iPhone smartphone, the iPad tablet computer, the Mac personal computers, the iPod portable media player, the Apple Watch smartwatch, and the Apple TV digital media player. Apple's consumer software includes the macOS and iOS operating systems, the iTunes media player, the Safari web browser, and the iLife and iWork creativity and productivity suites. Its online services include the iTunes Store, the iOS App Store and Mac App Store, Apple Music, and iCloud.