Challenges you might face if you are a data scientists

By arvind |Email | Sep 12, 2018 | 7353 Views

Being declared as "the sexiest job of 21st century" sure makes it the jobs that attracts people and if you have the right skill set you might be the next great Data Scientist. The demand for data scientists is plenty; it is still a job that hasn't been around for long. But like other careers that don't come without challenges; data science too has its own challenges to concur. 
This article is a dive to explore the challenges of data science that one might face in real life

Specialist not a generalist
Instead of knowing bits about everything try to focus on one thing. In data scientist who has speciality in a specific area is better than a data scientist than is trying to do everything. Mastering a specific area doesn't mean you deviate from the basics. Mastering basic knowledge is the first step in a profession you start, that is when you can whet. You need to learn swim in the shallow water before you go for a deep dive in the ocean.
Tal Kedar (CTO Optimove) said, "I would encourage new professionals to understand that data science is a bit like medicine, it is a vast and vague term that encapsulates wildly different practices under one roof". He further added "Data Scientists have very different engineering skill sets experienced with very different platforms and tools."

Understand the business and your choices
For a data scientist- your involvement in making things happen should not be limited to the 'how's' but also the 'why' of things. As a data scientist you sought to have a vast knowledge about the different business factors, using that knowledge to create a mental model that can be tested by your data, your job is not to sit there and look through the data to create connections.
Kedar explains this with an example, "If you are building a self-driving machine, you need to know what makes a good driver, and be well-versed in the challenges and outcomes that accompany safe or reckless driving, and then have those reflected in the algorithms that drive the car."

Having cross department expertise
If you have a different career background it can actually be a positive for you as a data scientist. "The best data scientists are not just statisticians or machine learning experts; they are also an authority in the field or business where they are applying those skills," says Kedar. Scott Hoover (Director of Data and Analytics at Snowflake) says, "Data scientists are arguably best utilized as the glue between technical and non-technical teams. As such, in addition to having a deep technical foundation, they must have a domain expertise in whatever department or area they are focused on, be it product, marketing, sales, or finance." Your unique background and blend of skills will be one of your greatest strengths.

The bridge between the technical and the non technical
Not everyone in the work place is able to understand the difficulties of one's work. As a data scientist who spends their workdays around technical terminology, this can be a source of frustration.  As a data team it's essential that you are able to communicate effectively with audiences of other departments who may not understand the complexities of your job. Hoover agrees: "A data scientist that cannot articulate what their model does and why it's of value to business stakeholders is going to have a difficult path to success." This is something you can practice. When you're working through a data problem, think about how you'd explain it to your family and friends (in a way that doesn't make everyone's eyes glaze over).

Working with raw data
As a data scientist, the primary challenge may be how do we use the data, including how to extract data, how to clean data, how to analyze data, how to get insights or build models from data. Data scientists should have extensive domain expertise in programming languages including SQL, Python and R." The majority effort of a data scientist has to do with creating a clean data set with useful information, all before any of the compelling machine learning or statistical models can be applied.

Working with other departments
Since multiple departments usually work together on projects, it's necessary to collaborate, compromise, and set clear boundaries and expectations. "A common challenge I face in data science is facilitating cooperation between departments on how data should be collected and interpreted," says Seitz. "Predictive models and historical analyses are only as powerful as a team's agreement on the validity of the source data." Engineering and data teams are often closely linked, so this pairing is also where misalignments commonly occur, says Sofus MacskĂ?ssy, vice-president of data science at HackerRank. "There needs to be harmony between the two, so engineering teams can seamlessly access data and engineer an infrastructure that allows the data science team to accurately collect and analyze quality data."

Flexibility and consideration
When handling a certain type of problem you should be open to different approaches, don't just wear blinders and run in a single direction. Flexibility to pivot based on unique situations is what will lead you as a data scientist to an optimal solution. An example, "You may use a recurring neural network when studying something that changes over timeâ??like the lifetime value of a customer. But you may opt for a Convolutional network when you need to extract features in an image classification task, like deciding whether a picture contains a dog or a cat. An adept data scientist will know all these approachesâ??and not be tied to just one and apply the one that best suits the problem he or she is trying to solve."

Maintenance And Control
As Seitz notes, small mistakes can be costly in data fields like machine learning by affecting your results. Catching them early is crucial. "Invest the time in refactoring your code, validating data sources and documenting changes with version control," says Seitz. "Hidden data dependencies, unstable data sources, and undocumented assumptions can lead to unexpected changes in your results when retraining models." The challenges of data science may be intimidating, but many can be averted with enough preparation and communication. As you learn how to become a data scientist as a beginner, keep them in mind and you'll have an advantage from the start.

Source: HOB