Short DescriptionApple is hiring a Hadoop Operation Lead Engineer who has experience with Configuration Management and automation along with an Organized, focused on building, improving, resolving and delivering.
- Hands-on experience with handling production clusters (Hadoop, Spark) as well as strong development/automation skills and being very comfortable with reading and writing Python and Java codes
- Overall 10+ years with at least 5+ years of Hadoop experience in large scale, multi-tenant production clusters (1000+ nodes with Queues) with a tools-first mindset
- You build tools for yourself and others to increase efficiency and to make hard or repetitive tasks easy and quick.'
- Experience with Configuration Management and automation along with an Organized, focused on building, improving, resolving and delivering.
- Good communicator in and across teams along with strong team management and leadership skills
- Taking responsibility for maintaining and scaling large scale (1000+ nodes) production Hadoop and Spark clusters as well as managing a team of Hadoop Engineers providing mentorship and allocation of tasks along with technical help required by the engineering team
- Strong knowledge of Kerberos for operation and debugging Kerberos issues in Hadoop clusters along with being the sole point of contact for Hadoop related critical issues coming from Application teams and internal clusters
- Being responsible for the implementation and ongoing administration of Hadoop infrastructure including monitoring, tuning, and troubleshooting and providing hardware architectural mentorship, plan and estimate cluster capacity, and craft roadmaps for the Hadoop cluster deployment
- Improving scalability, service reliability, capacity, and performance of the cluster and applications running in the cluster
- Triaging production issues when they occur with other operational teams, conducting ongoing maintenance across our large scale deployments across the world and writing automation code for running large BigData clusters
- Working with development and QA teams to design Ingestion Pipelines, Integration APIs, and provide Hadoop ecosystem services and participating in on-call rotation supporting the Hadoop Ecosystem
- Detailed experience in solving incidents, formulate theories and test hypothesis and narrow down possibilities to find the root cause
- Familiarity with geospatial data would be an added advantage
- Apple is an Equal Opportunity Employer that is committed to inclusion and diversity. We also take affirmative action to offer employment and advancement opportunities to all applicants, including minorities, women, protected veterans, and individuals with disabilities.
Hadoop Operation Lead Engineer