How and why to become a machine learning engineer
There is such a large interest in the machine learning industry we thought it was time to talk to some experts in the field. So, we sat down with James and Allen, founders of AdaptiLab. Educative recently collaborated with them to launch an extensive course training engineers with no prior experience in the subject to master Machine Learning.
AdaptiLab (Techstars Seattle ‘19) is a B2B startup solving the biggest problems in the machine learning and data science talent pipeline.
Artificial Intelligence (AI) and Machine Learning (ML) are two of the hottest fields in technology right now. From healthcare and agriculture to manufacturing, energy, and retail, many companies across industries are leveraging these technologies to get ahead. In fact, the Future of Jobs survey found that 73% of companies are likely to adopt machine learning by 2022.
With such heavy investment from companies, the demand for machine learning engineers and data scientists has already far surpassed the current supply. But there’s a silver lining: If you’re already a software engineer, developer, or programmer – self-taught or not – you can become a machine learning engineer faster than you think.
To explore this notion, we sat down with James Wu and Allen Lu, AdaptiLab’s Founders, to discuss the state of the Machine Learning industry and how engineers can build the machine learning chops required to compete in the market.
Why Machine Learning?#
One of the most exciting things about this new technology revolution is that any engineer can come up with new models or applications. Deep learning is the driving technology behind many meaningful innovations, such as self-driving cars, voice assistants, and intelligent robots.
As the world continues to come up with these exciting applications of deep learning, the need for competent deep learning engineers will continue to grow. Anyone with a base level computer science background can make an impact.
How did AdaptiLab come about?#
James and Allen have backgrounds in machine learning and have experience across academia, startups, and larger corporations like Google and Microsoft. After experiencing industry machine learning across industries and verticals, they noticed a trend: hiring for machine learning talent is incredibly difficult.
Not only are companies struggling to hire the machine learning and data science talent needed to stay competitive, but the process is time-consuming, inconsistent, and impossible to scale. As a result, most companies are unable to build their data teams and are ill-equipped for the data-driven era.
AdaptiLab is building a suite of services designed to help companies acquire quality talent. These include machine-learning driven automated technical screening, background-blind candidate sourcing, and industry-focused talent development and upskilling.
What inspired you to create a Machine Learning Course?#
Modern solutions and methods for teaching machine learning engineering focus on theory rather than practical application. Software engineers lack the opportunity to easily learn machine learning skills needed to thrive in the data-driven era. A lot of developers already have the foundation required to learn machine learning, but advanced academic programs and courses don’t focus on the practical skills required for the majority of machine learning projects in industry.
The Educative course is designed to teach engineers how to utilize industry standard Python frameworks manipulating and processing data: NumPy, pandas, scikit-learn, and Tensorflow.
Inspired to jumpstart your career in machine learning?#
Check out Zero to Hero in Python for Machine Learning These text-based courses are easy to skim and feature live coding environments, making learning quick and efficient.
Where do you think the industry is going?#
James and Allen believe within the next 5-10 years, a regular full stack engineer will be expected to be able to work with data in the same way that they’re currently working with web frameworks. With machine learning being at the core of what the world economic forum is calling the 4th industrial revolution, it will need to become a part of every engineer’s vocabulary.
There is a wide range of projects and domains requiring various expertise, but ultimately, there is still a huge gap in the supply of machine learning engineers and data scientists. As new companies entering the space are quickly finding out, there are two major concerns with the supply of talent: quantity and quality.
-
Quantity: Finding engineers and scientists with machine learning experience or formal training is like finding a needle in a haystack. Companies that want to be competitive are forced to interview quickly and make offers under extreme competitive pressure from other companies threatening to close the candidate first. Beyond the interview funnel, the sourcing of candidates is it’s own incredibly difficult challenge because the field and its applications are so nascent.
-
Quality: Oftentimes the top machine learning engineers and researchers migrate to well-funded research entities at large organizations, such as Google Deep Mind and Microsoft Research, or top tier tech companies with massive amounts of data and infrastructure, such as Facebook and Amazon. Others choose to found or join smaller tech companies, preferring the autonomy of the startup world. If they follow the latter route, large corporations like the aforementioned Amazon, Microsoft, and Google acquire those startups just to access their talent, creating a massive gap in the market.
What is a common misconception about ML?#
Let me preface this by saying there are absolutely difficult problems being solved across industries and verticals. However, there is a common misconception among the general population that ML and AI are an incredibly difficult and advanced field, requiring robust experience in calculus and statistics.
However, there were two major misconceptions that came up during our conversation:
1. You don’t need to learn the incredibly complex theory behind all the ML models. In the industry, 90% of projects don’t require you to build complex models. Those projects are actually data engineering, data analysis, and analyzing insights from models rather than the process of actually building models.
2. Companies only want to hire machine learning engineers that have academic training in machine learning. Because of the vast opportunity machine learning brings to the table, companies are going to be looking to hire engineers that have experience in software and data engineering. Meaning that as long as you can apply machine learning concepts to projects, companies aren’t going to require you to understand ML theory inside and out. Ultimately, developers who can apply those fundamentals to new applications and spaces are going to win in the end.
What does a software engineer need to know to get started?#
If you’re already an engineer you’re more than halfway there. There are amazing tools out there that can help you quickly learn industry level machine learning without having to dive deep into theory.
1. Understand the basics: As an engineer – self-taught or not – you need to have the basics of computer science down.
2. Have a firm grasp of data engineering: Again, as an engineer, if you have an interest in expanding your skills into machine learning, a large part of your role will require data engineering experience.
3. Start understanding the frameworks that exist: There’s a lot of great open source frameworks and tools out there right now. Less than 5 years ago there weren’t as many open source frameworks to learn from as there are now. Allen would have to spend hours messing with code and trying different functions, using the guess and check method. Unlike today, where there are open-source community-driven projects led by the major tech giants (Google, Facebook, etc.). These very new packages have contributed to practical machine learning, but they’re all very new.
4. Take a course designed to support your learning goals: As an engineer, you need to learn the techniques and frameworks required to transition to a data-oriented career. Start with the crucial coding skills and problem-solving intuition and once you have those, it’s easy to apply these learnings towards solving real industry challenges and building production-level models ready for deployment in an enterprise setting.
5. Find an industry or focus: One piece of advice James and Allen shared, is that engineers should pick an industry or vertical they enjoy. You’re going to spend a majority of your time analyzing data and if you find that data boring then most likely you’re going to hate it. Luckily, with the continued growth in machine learning, the possibilities are almost endless. Find something you like and dive in head first.
6. Go Deeper: If you are interested in research, then studying theory is going to be an important next step.
Continue reading about Machine Learning#
Free Resources
