How to get experience with scale as an engineer?

Question

Hey everyone.Feedback that I frequently receive from prospective employers is that I don&rsquo;t have experience dealing with scaling an app for many users, presumably concurrent users. This feedback comes from video streaming startups, primarily. I apply for full-stack positions, mostly.How do I get this experience without being in a situation where I have many users at my disposal?

davismwfl · Accepted Answer

Frankly, you can't get that specific experience without having a job doing it. Most everyone who has it today is because they grew with something from early on, or had a job interning or potentially did really well on the interview problems. I started on smaller systems that were distributed in the 90's then we went to large client/server then of course back to distributed solutions with the web. For me I scaled with a few different companies, the first problem I worked was initially small but went from 10's of thousands of users to millions of active users per week in like 6-9 months (dotcom days) so to me that was crazy stuff. Today what we did wouldn't be so remarkable but it was at the time and led me to keep solving problems like that.
I'll suggest a couple of things to try (not necessarily any order or preference). Do what most current people have done, get in on a small startup/company that hasn't yet scaled and scale up with the team. At the same time you should study up on scale problems so that you know the issues/answers even though you haven't dealt with them yet yourself. The study part will help with interviews until you have the experience to point at -- but even then you still need to understand the problems.
One other idea which is a little more involved, but pretty fun too. Create your own public experiment using one of the cloud providers where you build an endpoint/site that does some complex problem (pick something in the area you want to get a job) concurrently and then load test it and work through issues so that you can point someone to that repository as part of your learning experience. I have actually done this myself a number of times experimenting, not really for showing anyone but so I could understand issue(s) better. You might blow a few bucks (nothing crazy) on AWS or GCP/Azure to experiment but it will be invaluable for your knowledge.
As a hiring tech mgr when I am hiring for scale I look at people who have the experience first, those who have attempted and have the fundamentals second and lastly I'll consider someone smart who has the education (informal or formal) but lacks the experience if they have the enthusiasm and right attitude. I don't think I am unique in that thought process, maybe that helps you think through it.