You can build safe abstractions for manual memory management in userspace with LinearTypes. In fact Haskell already has plenty of manual memory management modules in its stdlib - it's just that they live in IO.
You can do the unsafePerformIO internally and expose a pure API made safe by some combination of LinearTypes and the ST trick.
LinearTypes guarantee that all values are used exactly once if the result is used exactly once. If an exception is thrown, the result won't be used once and the memory would leak.
One cheap way is to also track the memory with the GC. This is similar to using GC finalizers on file handles - closing handles via GC is inefficient and slow but it's only the last resort.
Another option is to use exception handlers to guarantee that the Ressource is freed. Either one handler per Ressource which destroys all advantages of linear types, or one handler in total and some mutable memory to track the still open resources.
You can do the unsafePerformIO internally and expose a pure API made safe by some combination of LinearTypes and the ST trick.