first of all, ahrefs discounts the "people" cost, but that's a huge cost to ignore!
the biggest advantage that AWS and the like confer is being able to reduce interactions with literally every piece of infra you consume from them down to APIs.
having physical hardware means you need a team who knows how to rack/provision/configure/update hardware *along with* administrating operating systems and everything that comes with *along with* the automation needed to hold everything together.
finding people who had all of those skillsets was super challenging before The Cloud appeared, and is especially hard now since everyone who would have those skillsets prefers to work with cloudy things (because everything's an API).
second of all, they made the classic mistake of doing a one-to-one comparison of running their business on EC2. ofc that's going to cost a ton! you're basically just renting VMs from them at a huge premium. that can be done anywhere else (Hetzner is popular) for much cheaper.
that's not why you move to the cloud.
when AWS or Azure says they help companies save money, they usually mean taking an app that runs really well on-premise on a fixed set of compute that's a whole process to scale and making it run even better on smaller, but more distributed, compute that should be less expensive due to economies of scale.
Do web crawlers like these _need_ to run entirely on huge EC2 instances that run hot all of the time? Could they take advantage of more fractional compute from things like EC2 spot autoscaling groups or "serverless" compute? Ahrefs uses local NVMe storage for everything, which is definitely cheaper than EBS. Could they use data archival pipelines to compact and move less-used data onto slow networked storage? Could they benefit from using more aggressive caching for sites that don't change very often?
finally, for every company like Ahrefs who runs lots of compute hot 24x7, there are at least 20,000 companies who spend big money operating datacenters for apps that don't justify the cost. they _could_ save significant amounts of money by moving to the cloud AND re-architecting their apps to spend compute more efficiently.
> first of all, ahrefs discounts the "people" cost, but that's a huge cost to ignore!
It's the hardest to compare. Cloud has a significant people cost too, particularly in the complexity. EC2, lambda, S3, RDS, IAM, cognito, glacier, light sail, EBS, fargate, cloud front, SNS, dynamo, elastica he, etc etc.
If you ignore this, you fail to completely understand a service you start to use. That's the quickest way to a surprise mammoth bill or insecure service.
> they _could_ save significant amounts of money by moving to the cloud AND re-architecting their apps to spend compute more efficiently.
This is a bad argument generally. If an org, institutionally, can't properly size services or they leave a lot of fat in their services.
Why would a move to the cloud fundamentally change that? You can even more easily over provision and overspend when it's a mere button click away.
I fail to see how a move to the cloud basically creates greater institutional ability to trim fat and optimize that just wasn't there before.
And I just love how convenient re-architecting is so casually thrown in like it's a walk in the park... Yikes. Rewrites can and do kill companies. Not a casual thing to undertake because a new vendor comes along with a shiny new tool.
> having physical hardware means you need a team who knows how to rack/provision/configure/update hardware
Not with a colo. They all have that as a service. If by provision you mean installing an OS via PXE or the like, then no, but everything else, yes.
> along with administrating operating systems and everything that comes with along with the automation needed to hold everything together.
If you have a team managing Terraform for cloud infra, congratulations – you have a team who can manage physical infra. Seriously, it’s the same tooling. Pick your favorite configuration management tool (Ansible, Chef, Puppet…) and you’re off to the races.
first of all, ahrefs discounts the "people" cost, but that's a huge cost to ignore!
the biggest advantage that AWS and the like confer is being able to reduce interactions with literally every piece of infra you consume from them down to APIs.
having physical hardware means you need a team who knows how to rack/provision/configure/update hardware *along with* administrating operating systems and everything that comes with *along with* the automation needed to hold everything together.
finding people who had all of those skillsets was super challenging before The Cloud appeared, and is especially hard now since everyone who would have those skillsets prefers to work with cloudy things (because everything's an API).
second of all, they made the classic mistake of doing a one-to-one comparison of running their business on EC2. ofc that's going to cost a ton! you're basically just renting VMs from them at a huge premium. that can be done anywhere else (Hetzner is popular) for much cheaper.
that's not why you move to the cloud.
when AWS or Azure says they help companies save money, they usually mean taking an app that runs really well on-premise on a fixed set of compute that's a whole process to scale and making it run even better on smaller, but more distributed, compute that should be less expensive due to economies of scale.
Do web crawlers like these _need_ to run entirely on huge EC2 instances that run hot all of the time? Could they take advantage of more fractional compute from things like EC2 spot autoscaling groups or "serverless" compute? Ahrefs uses local NVMe storage for everything, which is definitely cheaper than EBS. Could they use data archival pipelines to compact and move less-used data onto slow networked storage? Could they benefit from using more aggressive caching for sites that don't change very often?
finally, for every company like Ahrefs who runs lots of compute hot 24x7, there are at least 20,000 companies who spend big money operating datacenters for apps that don't justify the cost. they _could_ save significant amounts of money by moving to the cloud AND re-architecting their apps to spend compute more efficiently.