It's funny. All the examples they show in the blogpost are just things that were already pretty easy without agents. Sending an email when the CI pipeline passes, when a support request is incoming, when an order is shipped. I think we haven't found a problem for this solution.
The problem is how bullshit transactional emails are when you're outside of AWS. If you're not expecting to use 10,000 emails a month but would like the option to go over the free tier without committing to 10,000 more. Just let me pay per use FFS.
The reason AWS does that is because there is a lot of base level work to verify you as "not a spammer" and to keep verifying you. So this is their way of making sure you pay the base cost.
They could price per use, but it would have to start with a base fee that is about the same at 10,000 emails.
The base fee doesn't need to be monthly, they can take $50 dollars as a one time registration verification fee. That should be enough to cover their compute costs for the year, specially when you pool in that from multiple customers. Who is spamming with 100 monthly emails? How much compute do they need to verify you aren't spamming. They can bake all of that into the pay per use price, they choose not to and I'm glad cloudflare is offering this.
Maintaining the reputation of an IP address is the issue here. If one bad actor sends just a few emails that get marked as spam, the entire IP gets marked as bad. That's basically what you're paying for.
Also, the person who just wants to send a few 100 emails a month is actually far more likely to be a spammer. So it's also a way for them to eliminate those folks.
And lastly, the support burden can be high.
AWS has basically said they only want serious customers, let the other guys worry about the small senders.
I can't imagine that Windows 11 would be usable with a 2 core 1GHz processor and 4GB of ram. It might install, but opening the start menu alone will fill up your memory (slightly hyperbole)
> The direct corollary is that any successful compromise of the host can give an attacker access to the complete memory of every VM running on that node. Keeping the host secure is therefore critical.
> In that context, hosting a web service that is directly reachable from any guest VM and running it on the secure host side created a significantly larger attack surface than I expected.
It is kind of a fundamental risk of IMDS, the guest vms often need some metadata about themselves, the host has it. A hardened, network gapped service running host side is acceptable, possibly the best solution. I think the issue is if your IMDS is fat and vulnerable, which this article kind of alludes to.
There’s also the fact that azure’s implementation doesn’t require auth so it’s very vulnerable to SSRF
You could imagine hosting the metadata service somewhere else. After all there is nothing a node knows about a VM that the fabric doesn’t. And things like certificates comes from somewhere anyway, they are not on the node so that service is just cache.
Hosting IMDS on the host side is pretty much the only reasonable way to provide stability guarantees. It should still work even if the network is having issues.
That being said, IMDS on AWS is a dead simple key-value storage. A competent developer should be able to write it in a memory-safe language in a way that can't be easily exploited.
“No, there is another”—Yoda, The Empire Strikes Back :)
What you describe carries the risk that secrets end up in crash dumps and be exfiltrated.
Imagine an attacker owns the host to some extent and can do that. The data is then on disk first, then stored somewhere else.
You probably need per-tenant/per-VM encryption in your cache, since you can never protect against someone with elevated privileges from crashing or dumping your process, memory-safe or not.
Then someone can try to DoS you, etc.
Finally it’s not good practice to mix tenant’s secrets in hostile multi-tenancy environments, so you probably need a cache per VM in separate processes…
IMHO, an alternative is to keep the VM's private data inside the VMs, not on the host.
Then the real wtf is the unsecured HTTP endpoint, an open invitation for “explorations” of the host (or the SoC when they get there) on Azure.
eBPF+signing agent helps legitimate requests but does nothing against attacks on the server itself; say, you send broken requests hoping to hit a bug. It does not matter if they are signed or not.
This is a path to own the host, an unnecessary risk with too many moving parts.
Many VM escapes abuse a device driver, and I trust the kernel guys who write them a lot more than the people who write hostable web servers running inproc on the host.
Removing these was a subject of intense discussions (and pushbacks from the owning teams) but without leaking any secret I can tell you that a lot of people didn’t like the idea of a customer-facing web server on the nodes.
Of course, putting the metadata service into its own separate system is better. That's how Amazon does it with the modern AWS. A separate Nitro card handles all the networking and management.
But if you're within the classic hypervisor model, then it doesn't really matter that much. The attack surface of a simple plain HTTP key-value storage is negligible compared to all other privileged code that needs to run on the host.
Sure, each tenant needs to have its own instance of the metadata service, and it should be bound to listen on the tenant-specific interface. AWS also used to set the max TTL on these interface to 1, so the packets would be dropped by routers.
Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it. AFAI can tell, this service tells services running on the VM what SKU the VM is. But how is this useful to the service? Any Azure users could tell how they use IMDS? Thanks!
> Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it.
The "metadata service" is hardly unique to Azure (both GCP & AWS have an equivalent), and it is what you would query to get API credentials to Azure (/GCP/AWS) service APIs. You can assign a service account² to the VM¹, and the code running there can just auto-obtain short-lived credentials, without you ever having to manage any sort of key material (i.e., there is no bearer token / secret access key / RSA key / etc. that you manage).
I.e., easy, automatic access to whatever other Azure services the workload running on that VM requires.
¹and in the case of GCP, even to a Pod in GKE, and the metadata service is aware of that; for all I know AKS/EKS support this too
²I am using this term generically; each cloud provider calls service accounts something different.
Mainly for getting managed-identity access tokens for Azure APIs. In AWS you can call it to get temporary credentials for the EC2’s attached IAM role. In both cases - you use IMDS to get tokens/creds for identity/access management.
Client libraries usually abstract away the need to call IMDS directly by calling it for you.
Thank you, and everyone else who responded. So then this type of service seems to be used by other cloud providers (AWS). What makes this Azure service so much more insecure than its AWS equivalent?
Having it running on host (!), and the metadata for all guest VMs stored and managed by the same memory/service (!!), with no clear security boundary (!!!).
It's like storing all your nuke launch codes in the same vault, right in the middle of Washington DC national mall. Things are okay, until they are not okay.
I use GCP, but it also has the idea of a metadata server. When you use a Google Cloud library in your server code like PubSub or Firestore or GCS or BigQuery, it is automatically authenticated as the service account you assigned to that VM (or K8S deployment).
This is because the metadata server provides an access token for the service account you assigned. Internally, those client libraries automatically retrieve the access token and therefore auth to those services.
We run a significant amount of stuff on spot-instances (AKS nodes) and use the service detect, monitor and gracefully handle the imminent shutdown on the Kubernetes side.
> The incident also prompted LiteLLM to make changes to its compliance processes, including shifting from controversial startup Delve to Vanta for compliance certifications.
This is pretty funny.
The leaked excel sheet with customers of Delve is basically a shortlist of targets for hackers to try now. Not that they necessarily have bad security, but you can play the odds
I am not defending Delve or anything and I hope they get what they deserver but there is no correlation between SOC2 certification and the actual cyber capability of a company. SOC2 and ISO27001 is just compliance and frankly most of it is BS.
Sure it's certainly not perfect and a lot of the documentation is something you just write for the audit and never look at it again but that's why I am saying play the odds. The average delve customer startup might be less secure that the average startup who has to justify their processes to a real auditor.
Personally, I use them as frameworks to justify management processes.
A) I tie the cybersecurity activities to business revenue enabling outcomes (unblocked contracts), and second to reduced risk (as people react less to this when spending the buck).
B) with the political capital from point A) I actually operate a cybersecurity program, justify DevSecOps artefacts, threat modeling, incident response exercises, etc.
What this SOC2 reports, ISO27k certificates are, more like a standardization for communicating the activities of the org to outside people, and getting an external person to vet that the org doesn't bulls*t too much. but at the end of the day, the organization is responsible for keeping their house in order.
I went through SOC2 Type I and II. I’d say that most of that stuff is necessary, like splitting environments and so on. That doesn’t mean it’s anything close to sufficient to avoid being hacked.
It’s a framework to give you the direction, then if employees are careless (or even malicious), no security standard is complete enough to protect a company.
Not to be pedantic about the topic but SOC 2 is an auditing standard, not a security framework. It defines what you’ll be assessed against but it doesn’t tell you how to build your security program. You’ll find the prescriptive controls in real frameworks like ISO 27001, NIST CSF, or CIS Controls which do give you a structure for implementing security.
Some of it is, but things like "your stage/dev and production environments should be completely isolated from eachother" are valid and most tech companies get lazy on this front
It was never about cyber capability. It's a liability transfer framework.
If a service provider has a control that says "we use firewalls on all network access points, and configure those firewalls to CIS benchmark whatever", and a third-party signs off with "yes we checked, they have the firewalls, and they're configured properly", you now have two parties you can sue when a security incident caused by lack of firewalls causes you material damage.
Your org's cyber insurance will also go down if you can say "all our vendors have third-party attested compliance, and we do annual compliance reviews".
It might feel like BS, and I'm inclined to agree with you because of the security theater aspect. (For example, Mercor had their verification done by what appears to be a legitimate audit firm.)
But it's not useless. It still forces you to go through a very useful exercise of risk modeling and preparation that you most likely won't do without a formal program.
If your goal is to maximize your posture against cyber threats, spending your time on SOC 2 compliance with Vanta (or similar) is a waste of time if you consider the amount of time spent compared to security gained.
It's incredibly easy to get SOC 2 audited and still have terrible security.
> forces you to go through a very useful exercise of risk modeling
Have you actually done this in Vanta, though? You would have to go out of your way to do it in a manner that actually adds significant value to your security posture.
(I don't think SOC/ISO are a waste of time. We do it at our company, but for reasons that have nothing to do with security)
Probably the most useful aspect of SOC2 is that it gives the technical side of the business an easy excuse for spending time and money on security, which, in startup environment is not always easy otherwise (Ie “we have to dedicate time to update our out of date dependencies, otherwise we’ll fail SOC2”).
If you do it well, a startup can go through SOC2 and use it as an opportunity to put together a reasonable cybersecurity practice. Though, yeah, one does not actually beget the other, you can also very easily get a soc2 report with minimal findings with a really bad cybersecurity practice.
That's exactly what I've done in the past. We had to be soc2 and pci dss compliant (high volume so couldn't be through saq). I wouldn't say the auditor helped much in improving our security posture but allowed me to justify some changes and improvements that did help a lot.
It doesn't force you go through risk modelling because by now most SOC2 platforms have templates you just fill in the blanks and sign off. Conversely, the auditors are paid by the company, so their incentive is to pass the audit so the client can get what it wants.
Because there's no adversarial pressure as a check and balance to the security, and AICPA is clearly just happy to take the fees, it's a hollow shirt. It's like this scene from The Big Short. https://youtu.be/mwdo17GT6sg?si=Hzada9JcdIPfdyFN&t=140
As usual, it's only people that care that force positive change. The companies that want good security will have good security. Customers who want good security will demand good security.
Yes they may be a BS in certain cases, however its still better than nothing. They do allow the companies to consider the questions atleast instead of claiming unawareness and most importantly it facilitates the incremental improvement.
What they really should focus on is making those models more efficient. With them most likely losing money on inference (+model training + salaries + building data centers), I can't see why they would want more compute and more products, since more tokens spent is actually bad for them.
Did you really mean to say 4.5? Gpt 4.5 used to cost $75/$150 per million tokens input/output. And it did not even seem to be that good to justify that. I would not expect many people were using it, and I doubt that "expanding to india" was what killed it (if it was that useful/popular they would have kept the api, or keep it for higher end subscriptions).
If anything it should have been no1 in the "openAI graveyard" website.
India in this context is a synecdoche for scaling consumer vs Anthropic's more enterprise-y route, but yes that's pretty much why we didn't get 4.5 with reasoning. Without reasoning, 4.5 had no future.
From Sam Altman himself:
> We had this big GPU crunch. We could go make another giant model. We could go make that, and a lot of people would want to use it, and we would disappoint them. And so we said, let’s make a really smart, really useful model, but also let’s try to optimize for inference cost. And I think we did a great job with that.
4.5 scaled into a unified reasoning model would have been an incredible model. It beat GPT-5 on accuracy and hallucinations without reasoning (!)
It just wouldn't have worked for powering things like ChatGPT Go's rollout and loginless chatgpt.com, so they dropped it.
(And if you want, you could argue it's the compute crunch that didn't let them do both... but Anthropic had to make the same choices at the time and went in the other direction.)
This all sounds like pure speculation to me. GPT4.5 was ok but not spectacular. The whole marketing was based on "vibes" and how interacting with it "felt more natural" etc. If there was actual use case for this model, I do not see why it would not be just offered for higher end subscriptions or through API. Other expensive models at the time, eg o1/o3 pro, were not served in the free tier, but only in paid subscriptions and apis, but that one did have use cases, so they did keep it at the time, until they prob took a more unified approach with their models. So I do not see why they could not have done something similar with 4.5 if it was an actually good model.
And I am not sure that Altmat's statements are worth taking into account. His statements are more about marketing and turning things in his favour rather than speaking the truth.
It's not even just that- with component selection you have a handful of datasheets that give you (ideally) fairly truthful information about the device. You can rather deterministically look at these and compare them.
Regular consumer products? Good fucking luck. Anywhere an LLM pulls from is probably going to be mostly SEO'd listicles.
Paywall so I couldn't read, they probably mention this: Eating more fiber (to a point of course) will decrease your risk of colorectal cancer. Most westerners, especially americans, are eating about half the recommended amount of fiber. Could certainly be a contributing factor.
> Dr Adam Collins, an expert in nutrition at the University of Surrey, says the way the jabs work in the brain and body might explain why weight regain is amplified once you stop taking them.
They mimic a natural hormone called GLP-1, which regulates hunger.
"Artificially providing GLP-1 levels several times higher than normal over a long period may cause you to produce less of your own natural GLP-1, and may also make you less sensitive to its effects.
"That's not a problem when taking the drugs, but as soon as you withdraw this GLP-1 'fix', appetite is no longer kept in check and overeating is far more likely."
If that is true, and it makes mechanistic sense if I think about opiod withdrawals for example, then it is a problem regardless. If you have a stronger hunger impulse then before AND a lower body weight and thus TDEE, I can imagine that it would be pretty tough mentally to maintain your weight, even if you count your calories.
reply