Hard multi tenancy can't realistically be achieved in the same logical K8s cluster. And it is a moving target, which makes trying to secure it by admission controllers... not a great plan.
One needs to look into things like VirtualClusters to even begin to consider hard multi-tenancy with potential hostile tenants(https://github.com/kubernetes-sigs/cluster-api-provider-nest...). That is just about the control plane. It doesn't even touch the data plane.
How secure that is even with the extra layer, I do not know. Even in the VM land we have seen crazy VM escape exploits over the years.T
One needs to look into things like VirtualClusters to even begin to consider hard multi-tenancy with potential hostile tenants(https://github.com/kubernetes-sigs/cluster-api-provider-nest...). That is just about the control plane. It doesn't even touch the data plane.
How secure that is even with the extra layer, I do not know. Even in the VM land we have seen crazy VM escape exploits over the years.T