I work in a very fast growing SaaS company.
When the company was less then 10 programmers a few years ago, we ran very tightly managed systems. The test coverage was enormous and gave us great confidence. The feature selection was done carefully. Big tasks, very little frustration.
Ever since the company started growing faster, everything we have achieved just seems to be turning into this huge mess. The number of systems goes through the roof. The overall management of these systems became very poor.
Passwords are found in source repositories, servers and processes appear to be not monitored in production, the feature selection became a democratic vote between business people, writing tests is skipped way too often, code is not cleaned up before committing, merge requests are incomplete and what not. The data science team even refuses to even talk about their practices. They just SSH/RDP into their servers, install dozens random products, hack away some Python or R or whatnot, take a database dump here and there. No GIT, no docs, no tickets, no nothing.
In short: Many new hires in a short time frame brought a huge lack in good practice and craftsmanship into the company.
Should the "Head of IT" of roughly 50 people even bother with the all the many little details of the work produced? Do they usually work at higher levels of abstraction and simply hold the teams accountable if things to south? How might an IT supervisor actively prevent things from going south in the first place?
What are some traits of good IT/tech-leads, that you worked with?
I would live to hear your wisdom and stories on this.
Your "Head of IT" is most probably "forced" into dropping those values in favour of a "up & running" product.
In my personal case (as Head of IT), I got a commitment to security from the upper management. This allowed me to drastically reduce access to resources like DB and SSH access and "force" people to document their needs which helps understanding where "South" actually is. While enforcing rules, many double/tripple/quadruple works where detected and migrated and also some new "Specialist" where found that were very good at their specifically assigned work.
As a programmer, you and your team need to understand the companies goals and try to establish according rules/workflow for your daily business. For example: Your product is basically "done" and now management wants "Features". The "Features" Section of your products requires a clear interface for "plugins" and probably also a quick way to "prototype" new features. So you as a developer should find appropriate ways to optimise your workflow which might require to change your workflow for a specific section of your product.
Of course, this is totally oversimplified, but I hope you get the point.
Bye the way, this all requires Communication AND Understanding of all particpiants! Management need to communicate the way you are are heading and IT needs to articulate what is needed for the upcoming tasks. Be aware, most solutions are not found in a day. Complex Situations needs time to fully grasp and find solutions.
P.S. If communication and understanding is missing, you are most probably on a dead ship. Be prepared to drown (evaluate your market value) and don't get emotionally involved (Danger of "burn out").
If you can point to all the new features delivered by dev, supported by ops, and analyzed by data science... and you can say that these features brought in new business and invigorated old business, then yes all the negatives are worth it.
However if you can't, then it's all dead weight and standards need to be enforced, otherwise you are paying your employees to make your code worse.
I would say this happens all the time at big corps, but if it happens at a 50 person startup then it will be a big risk factor.