Short Description
Foursquare is seeking a Data Engineer who can develop and maintain our data pipelines using Hadoop, Scalding, Luigi, Spark, Mongo and more.Job Description
- Develop and maintain our data pipelines using Hadoop, Scalding, Luigi, Spark, Mongo and more
- Partner with the Data Science team to investigate and implement advanced statistical models and machine learning pipelines
- Identify and implement performance improvements across all pipelines
- Data investigations to validate assumptions or find the source of a problem
- Assist client support and sales with client integrations
- 3+ years of experience working with Hadoop MapReduce and/or other big data technologies and pipelines
- You have a solid foundation in computer science fundamentals with particular expertise in data structures, algorithms, and design
- You obsess over data: everything needs to be accounted for and be thoroughly tested
- You are constantly thinking of ways to squeeze better performance out of the pipelines
- Strong Java or other object-oriented programming experience or, even better, experience and/or interest in functional languages (we use Scala!)
- Experience with Scala, Scalding, Luigi, Hive, machine learning pipelines and model training is a plus
- Bachelors Degree or higher in Computer Science, Electrical Engineering or related field
Data Engineer