Posting on an alt account here since my main account’s profile clearly gives away the firm I’m working for.
So I finally landed a Machine Learning job after several interviews plus months of searching. However, I’ve been at my current job for about 4 months on the AI team at a large corporation and I’ve hardly done any actual machine learning work. When I first arrived, I was assigned a project that several members on the team struggled with and failed at. I can’t give you any concrete details about it but I can say that it’s related to NLP and we’ve only been given a measely 200 unlabeled examples to work with. And even if we were given more data, it’s vastly different and far more complex than the types of problems most NLP models can solve today (artificial general intelligence might stand a chance.)
Long story short, I ended up building a 100% rules based system with no ML which worked alright on at least the 200 samples they provided after months of tweaking. I also tried to get more data to work with but it took weeks of “playing politics” only to end up receiving the wrong type of data.
Right now, i think 4 months is still too early to judge a new job but I feel if this were to continue for more than a year, it wouldn’t be good for my career. Also management seems to have this notion that ML/AI is “magic” and that the “smart” people they’ve hired should be able to come up with a solution without needing any data . I’ve also had even newer team members who joined my project who later complained outright that they want to work on a “real Machine Learning” project.
So as far as I can tell, even though I’m a Machine Learning Engineer, I’m having a lot of trouble getting the company to provide me with the data to train my models on (and a ML Engineer without data is like a fish out of the water.) I also think 4 months is too early to be looking for a new job, esp since I left the one before this one after 1.5 years. What do you think?
Your story resonates with me in a sense that often in the industry you are given a task that is just unsolvable with current ML state of the art, especially in NLP. Sometimes you are given noisy unlabelled small datasets and expected to do do some AI magic, and sometimes people want their problems solved with a click of an AI button even before they can articulate what their problems are. Sometimes you have to resort to rule-based approaches and label lots of data yourself. It is true that almost any ML job is 5% building models, and 95% is data cleaning, labelling, building reports, marketing your solutions and educating.
However, when your projects succeed, and your solutions solve real problems, you can make your case for better centralized data collection and warehousing, more resources allocated to data labelling etc. If a company that you work at does not have a developed data culture, it is up to your team to educate people around and build that culture. So hang on, you maybe the person to make the company data-driven and become the Chief of AI or something like that
Tough position, I was in a similar place with my last job. Big car manufacturer, relied almost exclusively on a rule based system despite only talking about ML in the interview. You’re always going to get this to some extent - I’ve never had a job that was exactly what they said it’d be… And I’ve had quite a few! I even got lucky, after a few months I was doing some interesting classical CV work & then got to work on DL. They hired a bunch of other people, including a PhD researcher with years of experience & all put them in these glorified data entry positions.
You’re right in that staying there a while (>1-2 years) would be bad for your career. 18 months on your last job isn’t that bad with tech. Also you could simply leave this position out on your CV - remember HR/recruitment people are doing similar kinds of stuff to get you into the job.
Really what it comes down to is what are your other options? If you’re in NA then there are loads of other positions, other places not so much… I would suggest you at least apply for a few jobs - it never hurts to look & it’s always worth practicing interviewing. This way you will know with some certainty what your other options are so you can make an informed decision.
Hey, I emphathise with that. I don’t have much experience as an ML engineer in big corps yet, but I do have quite a bit of experience as a software consultant. Sounds like this is a communication problem, and something I have encountered way too many times (albeit not in ML), hopefully I can offer a suggestion or two.
It’s very often that the person who hired you is less knowledgeable or technical than you are. Quite often, that’s exactly why you were hired.
As the most technical person on the team, part of my job as a consultant would be to communicate to stakeholders what a piece of technology is, the circumstances it would take for said technology to work, and the circumstances for which it is unsuitable.
If the decision is to proceed with said technology, my job would be to then to communicate that we need x, y, and z for this to be viable, develop a plan and work with the rest of the organisation to obtain x, y, z, and implement the plan. How that works in practice depends on your organisation, and might be difficult or easy. But this is not a technical problem, but a communication and organisational one.
So, to translate that into your situation, I would work with other people in your organisation to obtain more data.
I know this thread is super old but I came across it and thought the below might be interesting to those involved.
I work in a tech company, Rokt, that optimises eCommerce transactions for companies like Groupon, Ticketmaster, GAP & Walmart. Our tech uses the first party data available throughout the transaction to recommend next best actions on a 1:1 level.
We are also hiring a senior manager to lead our optimisation and machine learning team either in Sydney or NYC. Feel free to take a look and share away!
