Original Tweet - https://x.com/sarmag77/status/1987482331683140014
Back in 2019, I was in charge of hiring 5-6 engineers for a new team.
This team was supposed to build automation around the existing APIs so that we could scale faster. The automation was supposed to be around all steps of the product, from CI/CD to production monitoring to automatic healing.
The entire product was used to manage 10s of thousands of SD-WAN routers in distributed locations with complex topologies. Building an automation tool for it was going to be a complex and fun affair. It may have been more complicated than building the entire product because it would require understanding the product completely and to build proper interfaces for data and actions which was not already exposed.
Being a company where there were only around 40-50 engineers, they didn't have a DevOps team previously.
So I put the hired engineers in the new team called the "DevOps" team. All of them protested saying that they didn't want to do manual work. That's when I realised that the entire industry hadn't settled on the actual scope of work that a DevOps engineer has to do.
In some companies, DevOps engineers do only manual stuff, basically checking logs, deployments, creating pipelines, etc. In some companies, they build the entire infrastructure around managing the product efficiently.
I believe that's where the distinction started coming in. The ability to build a system or platform to manage a product compared to just writing ad-hoc scripts or yaml configurations.
Today, there are different specialisations of platform/infrastructure roles.
Let's take the example of Stripe. Stripe hires specialised engineers for it's infrastructure teams. There are different infrastructure teams for different platforms, for eg: Mongo, Elasticsearch, Ruby. All these teams build tooling and platforms around these tools.
The advantage of these teams is that these teams are highly specialised teams whose main focus is to standardise and scale the overall platform so that the product engineer building features wouldn't have to worry about whether the data is being backed up properly or if there is insecure code being written.
Have the manual DevOps or Platform jobs gone away? No. But are they decreasing. Yes.
So if you are someone who is looking to become a platform or devops engineer, better focus on building specialisation on a specific part of the process.
I worked for database specific teams in the last 2 roles and it has always been exciting to understand what affects a distributed high throughput database from scaling.