* There are no solutions, just tradeoffs. Don't think there's a perfect tech stack.
* Done is better than perfect. Velocity matters. This generally means pick something you know if delivery is important.
* Boring tech tends to be reliable. Things like PHP, Rails, Django, etc. These tools have been around a long time and ironed out the kinks. Lots of documentation and tooling to make your life easier. Odds are very low your new fancy idea has any requirements that boring tech can't deliver on. This counts for every layer of the stack (frontend, DB, OS, etc).
I think the most important thing to keep in mind is you'll never have a perfect solution and that's ok! Don't get fooled by all the hype from new technologies that try to make you feel inferior for using other tech.
Decisions about architecture, code structure, algorithms, data structures, etc. are an inherent part of software development. That's not a bad thing per se. And yes, you will experience fatigue - it comes with the territory - it's partly why we get paid the "big bucks" (for some value of "big"). The best way to deal with it? Take a break. Go for a walk. Get away from it for a bit. Soften your focus. Take some time to play - write some fun code for a personal project where it doesn't matter if you make a non-optimal choice. If need be, take a sabbatical for a few months or even longer if you're experiencing burnout.
(30+ year Software Developer)
For experimentation and prototyping I tend to check out new things, those might end up in the 'boring' stack once I'm familiar with them.
For me it's a personal stack of tools that work for me, and I try not to be too influenced by what's the next new thing on sites like StackOverflow or Hackernews, I use what works for me and what makes my clients happy.
So I use Rails+React-on-TypeScript on Heroku for a web-app. I use Scala+Spark on Databricks-on-AWS for data wrangling. I use Python for local scripting. That's because in these domains I have mastered my tooling and that's way more valuable than using the best tooling that I have not mastered. The program design choices are similarly non-decisions. Just always do the same thing.
I do experiment for fun because it is enjoyable to me, and that's the mechanism to add more things to the proficiency list. For instance, I like Rust (and use forked Rust utility programs for personal use) but when I write my next command-line tool I use Python or Go. When I become proficient in Rust, it may become a possibility that I use it.
¹ Of course, the overwhelming contributor to product success is the product choices not the engineering. Few products succeed or fail on the engineering. In each case, you only want to innovate on the differentiator. It's like a variant of the "optimize the bottleneck alone".
There's time and place for using cutting edge solutions. But if that time is 'always' and that place is 'everywhere', you might be doing it wrong.
Keep things simple.
This is classic perfectionism.
When suffering from perfectionism, a person overweights the consequences of arriving at an imperfect solution and underweights the consequences of not arriving at any solution at all.
In other words, it becomes safer to exist in an undecided state where all options are on the table than to make a decision and potentially deal with regret later. We tend to think of perfectionists as people who tirelessly work until the problem is solved to perfection, but many perfectionists simply get stuck in decision paralysis. If you never make a decision, you never have to deal with irrational fears of picking the imperfect solution.
In reality, most of the time it’s better to pick something and run with it as fast as possible. Worst case, you have a good but not great solution. Best case, everything works just right. Either way, it’s better to have some solution than nothing at all.
Perfectionism is a classic killer of ambitious startup teams. Put a lot of smart, dedicated people on a problem and they’ll want to produce something they’re proud of. They want to collect all of the best practices and state of the art tools, as only the “best” will do for their work. Good intentions, but while they’re on their second rewrite and spending weeks rearchitecting to 100% test coverage, their competitors are running circles around them by shipping product that is good enough, even if it’s not ideal or perfect. Mediocre products that ship are better than perfect products that never ship.
If you find yourself struggling with this, there is a large body of work and self-help materials for dealing with and overcoming perfectionist tendencies.
* Stick with what you know. I personally know Python best, so if the solution can be achieved in Python, I go with that.
* Avoid fads. I don't use a language until at least two years after I've first heard about it.
* If you've gotten this far in the list, you'll have to learn a new language or brush up on a weak one, so look for one with a strong community and a lot of libraries.
* Old usually means stable. If it's an old language but still in active development, they've probably found most of the worst edge cases by now. Also there is a lot of best practices.
* Everything is a tradeoff. Nothing is the perfect solution, so know where your solution might be deficient (or learn).
* Doing cool tech is cool, but make sure you're solving the business problem (or personal problem) you set out to solve.
If you have a big work/production project, go with what you know, whatever that is. When there's a deadline or someone will lose money with a delay, it's irresponsible to use that project to add another line to your resume or play around with $NEW_TECH.
If you want to learn a new technology, do it on a project where you can afford to throw your hands up at some point and scrap the whole thing, because you very well may end up in that position.
If you try to support all the features that could be supported, you will end up buried in the bottommost layers of the tech stack, because that is how these things always turn out.
If you try to make life as easy as possible, in contrast, you most likely end up sucked into the Web stack and limited by what the browser does.
If you try to make the project "minimal" you may end up boiling the water into an empty pot - it does nothing because it wasn't trying to do anything.
But if you have other principles in mind as you start the project you can evaluate the stack in terms of whether it facilitates study of each principle. "Minimal" is just the principle people tend to grab when they lack for others - add others and minimalism has the intended clarifying effect. There are still tradeoffs to that, but they are much less deleterious.
What other principles can be applied to software? You have many options: Ethical concepts, of course, center the good for society. Cosmology often has ideas for the specific design paradigms. And artistic ideas like improvisation, remix, and so on suggest the mode of use. Most software tends to assume Taylorism by default: Automating for speed and productivity along a few measurable axes. If you relax that assumption you can often streamline the features.
In any case, having a set of principles in hand lets you shop for tech purposefully: From principles you end up with rules, and from rules, specifications. That makes evaluation easy.
A better approach to solve for this problem is to completely ignore, at least for the moment, any idea of decision fatigue and instead focus on developing a solid plan. A good plan will account for risk acceptance, business requirements, a task list, and costs. The most valuable part of a solid plan is not having some magically complete roadmap or blueprint, but that it puts all your challenging decisions up front. It forces you to examine what you are willing to pay in terms of money, time, and effort.
Once you have a well envisioned plan the only consideration of decision fatigue should be limited to maintenance. If a given maintenance effort is too expensive or requires too much investigation or risk then you need to refactor and break things down into more primitive units.
When it comes to running a team there are only two things to consider heading into planning: a defined performance baseline and scope of effort. Those two things are completely in competition and you have to balance them against a budget of available resources. As the team leader be willing to accept input and suggestions from your team, but set the baseline with confidence and don't let the children run the daycare.
The fear of making a bad choice paralyzes many people. So let me offer a little help: Do not worry you will get it wrong, you will get it wrong. Invariably there is something better you have not heard of. Just aim for the top 25%.
Here are a few things that I have learned:
# Requirements - List out clear requirements, get an agreement with your co-decision makers, and the necessary people. Make sure you address for at least 2-3 yrs of requirements. It is likely, that new requirements would pop up and your system should be adaptive to this - if you think that your system is not going to address a specific thing, think twice, you may get buy-off & assurance from product, directors. However be prepared mentally that six months down the line, this will change.
# My Dos and Don't (always subject to change :))
- Do keep things simple
- Don't make a career out of a single project.
- Do list tradeoffs in your design
- Do document and perform deep dives.
- Don't ship prototypes.
- Don't be afraid to get critical feedback. If someone gives critical feedback, spend a few days to clearly understand the feedback
- Don't be defensive
- Don't be attached to what you build, it does not represent you.
- Do use new tech in baby steps
- Do experiment - The more you experiment, the more you learn, the more you will fail. Is this something that you have the mindset for? You can also ask how much do you have the time/energy for?
Reversible decisions should be made quickly and emotionally cheaply. Draw straws if you have to.
Irreversible decisions should be delayed as long as you responsibly can. The extra data helps reduce speculative arguments, gives your solutions a little more time to mature, or may discourage you from even tackling that problem at all (changing your goals or finding partners)
There are a thousand and one frameworks. You're going to pick one and you're going to learn it. And you will get on with your life. The only thing that matters when picking a framework is whether you feel motivated to learn it. Figure out what motivates you: is it a popular framework with lots of users so you have plenty of people you can ask for help? Is it a framework that you have to use because your IT department controls what infrastructure or languages you can use? Don't spend too much time worrying about which framework is best. That's not your job: you're translating requirements into code. Get that done and people will give you praise and money.
Game development? Use an engine and framework. Make the game. Same deal.
Where it's a bit different is when you're building the infrastructure and tools. Not too many people get to do that. Keep your scope small and don't try to be all things to all people is my advice. If you're writing libraries: make it do one thing. If you want to write frameworks then that's all you're doing is making choices on behalf of your users so they don't have to: so make good choices, make them the default, and prefer convention over configuration.
Update: clarified opening paragraph, don't want to come off as gate keeping!
At some point you need to commit. It's like relationships. How do you choose a partner out of billions of options? You browse around, learn what you like and don't like, make judgement calls, play around with some and eventually either make a conscious "this is the one" decision or else make a "we've been together so long, I guess we're a couple now" decision. At some point it becomes formal and you stop actively looking at other options because you're content with the option you chose.
If the system is all completely new to you, then pick a very small project (or wait until you have a lot of time on your hands and failure is an option).
You should look a new tech as something to be experimented with and tried out in a casual sense. New JS Framework? I'm going to try and re-create this thing that I know how to do in the Old JS Framework and compare. New database? Make a simple app to explore it and flesh it out.
If you don't know enough about the new product to compare it to what you know, you shouldn't be considering going with something new. New stuff comes out all the time, but that doesn't mean it's good or worth looking into.
How long did it it take carpentry to get past the great debate between "hammer vs screwdriver" ?
Will anyone ever refer to the saw as a "boring" way to chop wood ?
Should there be resume screenings for gardeners with "x years of experience with the acme showel" ?
Would the world be a worst place if you could just use a decent language with a reaonnable GUI library to store bits in a working DB ? (None of which I'm probably skilled enough to build...)
Maybe we would have to write software that actually works for a living, and we would be played much less.
Keep everything as simple as possible.
For business purposes when it really comes down to it, the tech decisions don't matter. Sure you could choose the wrong technology for a problem but because there is a wrong option doesn't mean there is only one right option, there are probably many right options with different trade-offs.
For example, if you have data with strong, well defined relationships, don't pick a data store that is NoSQL, pick a RDBMS. Does it matter which one? Not really, it matters more what your people are most familiar with since that will go a lot further. Pick based on your non-technical constraints - cost, skills, etc.
This all changes if you are tinkering on a hobby project to learn something. In that case, pick whatever you want. That might help alleviate making boring tech decisions for your employer.
At the end of the day however, you just need to pick something and roll with it. If it turns out not to be the best choice, you'll have at least learned something and the next time you have to choose you'll be better informed.
Don't stress out about it too much. It's a natural part of software development. When you are coding (after selecting all your tools), you are _still making tons of choices_. It's just in a different context; how will you structure your code? What coding paradigm will you apply? What algorithms will you use to solve your problems? You learned to cope with this stress too. It just takes time and experience.
Alone you are going to make bad decisions, if you have friends you can learn from THEIR mistakes and bad decisions, and their good decisions too, much faster than you alone.
Those guys have already made databases, they have built web apps or whatever you want to do.
If you know successful people on your field, use whatever they use, just copy them, correcting for your own circumstances and capabilities.
It is probably the best way humans(and primates in general) learn, by imitation from the best.
This can and is hacked by advertising industry because it works subconsciously: You see the best tennis player in the world drive a particular car model or wear a watch and you want this car model or watch. Or they watch their favorite musician take drugs or get tattooed and want to do the same.
This makes no sense(copying some master outside her area of expertise) but if you are into tennis you should "spy" what the best tennis player does for training and so on.
What is incredible is that most people will help you if you just ask them.
Once you do you will learn:
* It is not overwhelming. There are very clear tradeoffs with each technology.
* There are not so many options available that are mature and proven.
This doesn't necessarily help the original decision, but it does stop a bunch of re-litigation of previous decisions, and this is often a cause of noise and distracting headspace. Importantly, it can remind everyone that the decision was made with a good reason at the time.
The inferior choices you need to worry about are things like poor architectural descisions, or bad coding habits. These are the things that will "bite you later and trigger a rewrite". Not the stack itself.
I wouldn't personally like to work with PHP (my first language...), but I don't see the problem with that, if you do great work. Personally, I think that Python, javascript, and c++ are popular enough that they are solid choices.
Afterwards, it's a question on if you're making a choice for yourself, for the project, or to work with others. In any case, if you have some sense, you can learn given some time.
And I think if you're working on a personal project, you probably just want to see it grow. So, have fun. I'm sure you can re-write the prototype later.
Naturally he's not talking about choosing deliberately bad technology, so much as he's saying to lean toward choosing crude tools for most tasks, and only opting for sophisticated tools when experience shows you need them.
What I particularly like about this idea is that it short-circuits the idea of even wanting to have nice things, so that you can get straight to solving the problem in the most direct way possible. If your dinky solution doesn't work, it's probably also fairly easy to replace when the time comes.
For example, it's easy to slurp CSV files into almost any database, and code that relies on CSVs for storage probably has a simple enough interaction model that it's easy to migrate it as well. So picking CSV might make it easier to get to work, and save tomorrow's problems for tomorrow. But migrating from PostgreSQL to Cassandra, or vice versa, is a terribly daunting change. So you really want to be sure you're making the right decision on the first try. It's really easy to let that knowledge breed analysis paralysis - and it's arguably even irresponsible not to let it do so.
I have similar thoughts on programming languages: I'd rather default to the crummy old one, because newer, more sophisticated tools are much more likely to be able to easily speak to code that's written in a crummy old language, than to be able to link to each other.
The overall article isn't really about the question at hand, but some of Yossi Kreinin's thoughts here also seem pertinent: https://yosefk.com/blog/redundancy-vs-dependencies-which-is-...
Every choice is going to be inferior in some sense. If you can make a choice at all, consider it a good thing. Participate in the mess. Choose what makes you happy.
Paul Graham was onto something when he recommended having friends in programming [1]. Use what your friends are using.
https://www.kabdebon.com/cto/2020/06/20/making-boring-techni...
My two cents, if you want to build a business or a project for a company: go with battle-tested, mature technologies, always. Take away the risk of immature technology getting in the way of the only mission: delivering value to your customer
Now for a personal project... go crazy, have fun and try what's new out there.
Don't feel bad about letting tasks slip. The initial task is to learn what you should really do. If it's new development, this can take quite a long time.
Don't estimate the happy path. Yes if you already knew how to do it, it would take 2 hours to write the code. Estimate for the learning and the unhappy path.
Six hats is kind of dumb, don't take it too seriously, but there is something useful there: Plan to spend time planning, doing, testing, task managing, communicating. Like really plan for it.
Once the task is checked off, I can mentally free myself from the burden of thinking about the other options. If the decision turns out to cause unforseen problems, I can revisit my notes on why I made the decision and reevaluate with the new information.
https://github.com/andrewt3000/web_development#web-developme...
I think this site leans pretty niche.
My approach to this is recognize why its happening.
Its not the "dizzying array of choices" that is causing the indecision, its your thoughts about the array of choices that is causing the indecision.
The "dizzying" aspect of this is what you're brain is adding to the neutral array of choices. If you continue to think about it this way you'll be stuck in indecision.
Recognizing that is the first step.
For situations where there is not a clear popular or technical standout, I timebox the research time. After x (days|hours) I need to have a decision spec'd out. That way if I spin my wheels too long, then at some point I hit a deadline and my time is up!
When you only get paid for creating solutions that work or your earning capacity is reduced when you spend too much time dealing with broken systems, it makes these decisions very simple.
This is the exact same in code, start with the language and the tools that you and your team know.
When you start a project, a new feature, a new anything, you should always start with what has worked for you in the past.
When you start a new project, instead of viewing it as "I need to do X", instead view it as, "I need to do X using Y." You're not trying to write the best web app ever, but rather you're trying to write the best web app using Coffee Script and React.
All mediums have disadvantages and restrictions. Watercolor may lot look as realistic as oil paint, but the watercolor painter isn't concerned with that. The watercolor painter is concerned with watercolor and watercolor alone.
I say this because some of the most fun I've had programming was writing assembly for 8-bit retro computers. In terms of productivity, choosing a shitty CPU from 1980 as a target is not productive whatsoever. But that's not the point. I wasn't trying to achieve the global optimum of success. I was trying to achieve success given a list of restrictions set at the beginning. And that was a more attainable, healthy goal.
So pick a choice. Pick whatever you think would be an interesting challenge. And stick with it. It's your medium now.
If you don't know any tools, pick one and learn it well.
Instead I would start jamming on a whiteboard or in a notebook. Think about the problem you are tackling: what are the inputs, what are the outputs, what are the key functions, and what data structures and algorithms might be important? How about the team? Is it just you, or are you working with other people? What are your experiences, interests and passions? What's the timeline? Spend some time exploring libraries, reading paper and blog posts to inform yourself of the space. Maybe read some open source libraries. Don't go nuts, but scan around to soak up some good ideas and build a mental model of the space. This all helps put some parameters around the situation and give you some confidence that you are making informed decisions. Now you can think about the MVP, key interfaces, etc, and you might find that you can do some quick and dirty components for 80% of the project while really focusing on the heart of the challenge in that last 20%.
Often times if you define good interfaces you can swap early hacks and experiments out for something fancier (faster, more scalable, etc) later on, so then you don't need to worry as much about making optimal decisions. For example, you might use some distributed key value store or message bus in the end system, but to get the ball rolling you can just spin up a Redis instance during development, or even just write a couple functions that fake it in-memory or using JSON on disk for starters.
Now you can think about what you want to learn in this project. What's going to be fun and interesting about it? Do you want to dig into some new algorithms, a different language, a machine learning model? I like to use my curiosity as a major motivation to differentiate from other solutions and push through what could otherwise be boring aspects of the project. Say you need a service that just manages logins and sends back some tokens. That can be done in a million ways, and it is likely to get swapped out down the line anyways as you scale or whatever. So make it interesting, pick a new library or language and go nuts. This of course needs to be balanced with making sure you can accomplish the task in a reasonable amount of time, but this kind of learning is empowering and it will make you a more capable engineer. Now 20 years into my career I can program in 10+ languages on embedded micro-controllers or virtual cloud clusters, and picking up a new language or framework isn't daunting at all.
Have fun doing it, and you'll get better faster than everyone who is picking the boring framework in their same old language to solve it the way that the blogs are telling them to and then frantically searching stackoverflow because half way into the project they discover that their square problem doesn't perfectly fit into the framework's glorious round, ahem, hole. That's not engineering or craftsmanship. That's nervous fanboy frenzy that is the opposite of empowering.
Everyone has to find their own explore vs exploit trade-off, and we need people whose experience is narrow and deep as well as others who have learned far and wide. Choose what kind of person you want to be and make it happen.
If you read the above, you might get the impression that I'm saying people rarely change stacks. And that's actually true. At the end of the day, people tend to stick with what they know, because it works, and has worked for years. Rewriting software is risky and expensive. But even for new projects, if the teams sticks to their current tech stack, they can reuse years of work, as well as rely on the expertise they acquired through the problems they faced. Now I know that this kind of question often comes from people that are either new to the field or not very experienced. So I will give some advice if you get a fresh start, as in a start up with no legacy code or existing team, or if you are building or own project, or maybe you get a team that is somehow isolated from the rest of the company.
Basically if you have no real world bias (as opposed to personal bias) towards any tech stack, then the only solution is to do a lot of research. Building prototypes often help test out the options. But first learn about them. Learn their motivation. Why was this stack built to begin with, what problems were they trying to solve. Are their problems similar to what you expect in you project?
Sometime people build new stacks to address real world problems. Some times they build them because they can. I'm not saying you shouldn't use a technology just because the other had no good reason to built it, just had to opportunity. Sometimes people build really good products even if they are not trying to solve world hunger. And if you find such stack go ahead. But in general, try to find stable, battle tested tech, that solves your particular problem. Ignore the trends, focus on reliability and familiarity.
So Adam Savage had a video[1] a few months back on weathering prop money to look like a properly “gangsta” stack of bills rather than a set of nice printed paper. And when it was broadcast it was live and he took questions from the audience while some espresso was drying on the bills in his oven, and one of those questions was about someone needing a prop that looked properly covered in dirt and blood, and what is the best way to do that?
He said that the dirt part was easy (I assume he meant, just bury the stuff for a day or two and come back and it will be covered in dirt) but the blood part was harder because blood doesn't stay red for very long. But at the end he had some really great commentary, looking back at the question he said
> So, your question though, “what is the best way”... The answer is, there is no, there is no best way! There is the way you figure out how to do it. Yeah! That’s really the thing. It may be that you’ve gotta, like, take a brush and kinda work at a whole bunch of different bills, and then take the stack and then effect the stack, right?, I think that’s like a multi-stage process, and then you should do some tests with different paints and see, like, under the lighting conditions these will be seen—I don’t know if it’s for a theater or a film—does it communicate? Does it tell you that it’s bloody, dirty money? That’s the question to be asking, you gotta hold it up, look at it, [mimes looking at it and pausing and thinking] “does that sell to me?”—and ask other people, “does this sell?”... and like most of the time they’re gonna go, “no,” and you’re gonna keep on going. But it’s trial and error.
I present this advice to you in part as a way of combating decision fatigue. It’s gonna be trial and error, there’s not gonna be a ‘best way,’ so spending too much time up-front trying to figure out what is that best way* is gonna be a waste of your precious limited resources of attention and drive.
Because the actual definition of “best” is coming from these extrinsic factors. Sure you don’t have lighting. But maybe you have a constraint that you would like to build a JavaScript game around the same rules engine as you’re using to analyze the game and figure out cheap play and winning/losing, and it would be really tremendously inconvenient if you discovered that the two rules engines are subtly different and your analysis misses a loophole that breaks your JavaScript game. So you write the analysis engine in Node.js or so, so that you can reuse identical code in both places—or maybe JS is an inconvenient language because it does not have a great library that some other language does, like maybe you need to do a Fast Fourier Transform as part of this game so now you have to switch to a client-server model with the server having access to your FFT library. I don’t know. But the decision is made not by which technology is intrinsically better, just like the decision of which red goes into your blood paint is not about which paint is intrinsically better.
Two minor tips,
(1) I commented recently in another post that you should not concern yourself at all about scaling problems. I want to repeat that here. On initial selection of technology, do not worry about whether it scales. On initial selection of algorithms, it is even acceptable to brute force through some n! number of ways of permuting some elements until you find some better heuristic. Always start with doing it whatever is the easiest way for you personally, because you do not know until you see it in its eventual lighting what parts will and won’t work. It may be that this n! is indeed too long, mostly n is small but every once in a while it goes to 12, 13, 14, or even 15. But it may also turn out that the final decision can be effortlessly cached and then it becomes O(1) after that, so it is a one-time cost that doesn’t need to be recalculated. Or maybe it does. Spend all of your time optimizing that, and you may find out that n is never more than 8 in practice and the thing completes in some fraction of a second. You don’t know how to choose tech or algorithms to scale until you know what your scaling problems are, and when you know that you’ll be able to measure them. Never start with Kubernetes, never start with anything intricate, just do the easiest thing which might work and then see why it doesn’t. And this solves a lot of decision fatigue.
(2) In addition to ease, you might consider joy. Start writing the thing in Haskell, if that will bring you joy. You may find that with a Data.Map.Strict and a System.Random.StdGen that you can very easily load a dictionary of names and frequencies into Haskell and then randomly generate a name according to that frequency table, and maybe that is fun to you, that this fits in just a couple lines of code. If you are playing then you aren’t nearly as subject to decision fatigue. Nobody sits there with a box of lego trying to make a castle going “but is the 2x4x3 brick a better material to build this castle rather than say the thinner, longer walls of a 1x6x3 brick? What about a 2x4x1 brick, a larger number of thinner layers in case I need curvy designs built into the walls?” Like those decisions do have to be made but you kind of just make them and if it’s wrong you unmake them later, and you don’t care because you’re playing around and so it doesn’t feel like a big decision to you.
Those are my three pieces of advice. There is no best way there is just the way that you figure out which does the thing; do whatever comes easiest to you and worry about making it better later; and feel free to play around with other things you’re curious about.