HACKER Q&A
📣 distsysdude

How are distributed systems as a field of study compared to ML/AI?


I currently work on Database Development at Oracle. I would like to acquire knowledge on something other than the DB internals, and I've been recently fascinated with Distributed Databases like CockroachDB, AWS Aurora, Azure Cosmos, TiDB, etc.

So I was thinking I'd dive deep and learn more about distributed systems and possibly try to switch to any of the above mentioned companies.

I would like HN's opinion about the scope for Distributed Systems(& Distributed Databases) as a career choice, compared to the current hot areas like Machine Learning/AI.

A bit of background about me : I did my undergrad in a Non-CS field, but I've had an interest for Computer Science since High School, so the switch to a full fledged CS job was not that difficult. I'm self motivated and willing to spend copious amount of time to learn something that would help with my career. But I'm confused as to what to study.

Should I stick with my current interests (or) follow the industry trend, and try to learn an ML/AI ?

Thanks!


  👤 montereynack Accepted Answer ✓
I’m biased as my work is primarily focused on large-scale distributed physics simulations, and incorporating machine learning into these. As a result, I treat ML very much as a means to an end.

Of course with the caveat that your situation is unique to you so I can’t give any definitive answers, I would think long and hard before jumping on the ML hype train. In my experience, it doesn’t pay to follow the trend; you’ve either gotta be first or you gotta be unique. Now that’s not to say that doing ML work work will only be restricted to a select few which you aren’t a part of, but myself and a few others are wary that the ML hype train (at least as far as deep learning is concerned) might be passing. The days of the AI labs paying million-dollar bonuses are nearly gone, unless (and someone can correct me if I’m wrong) you’ve got an alternate skill set they’re looking for. Of course, that doesn’t mean there aren’t plenty of people and businesses who would need CRUD-type ML setups; with your experience in databases I imagine that could be a unique angle to attack it from. Whether it’s a good idea to try and pivot into a career using ML really depends on your specific situation and the opportunities therein; to get more solid advice I would ask a trusted colleague or mentor, and would not consult people online, even if they are from HN.

For my PERSONAL opinion: I can’t speak to what is normally done in other parts of distributed systems, since scientific tools are usually bespoke and don’t use the same set of approaches as commercial products. However, just thinking about it from an outsiders POV, it seems to me like focusing more on distributes systems would be a winning combination. I don’t think computers will advance enough in the next 30 years that the need for distributed data and compute management skills will go away; hell with IoT you might be looking at a boom down that career path. From my perspective it’s only upside if you focus on expanding your skilllset in these areas; if ML continues to thrive there’ll most definitely be a need for distributed systems to run these models on. And if an AI winter hits, you’ll have a solid set of core skills to fall back on which I don’t imagine will go out if favor anytime soon. Those are just my two cents though, of course YMMV.


👤 s1t5
They're basically everywhere, there's a lot to learn, it's a valuable skill for a potential employer and you're interested. Seems pretty obvious that you should go deeper with distributed systems. Also, your lack of formal education won't hold you back as much as it would with ML. The idea that you should go do math for the next couple of years just because the field seems hot, doesn't seem well thought out.

👤 drallison
Huh? Seems to me that you should be interested in and learning about all manner of computer related things including Distributed Systems, Machine Learning, and Artificial Intelligence. Knowledge and techniques useful in one context are generally useful in other contexts. The ability to see structure and similarities across systems and applications is invaluable.