I didn't reproduce your issue on my side. I am using Microsoft graph explore to test the api.
When I accessed the final nextLink response, I got the deltaLink.
Then I removed two users from a group on Azure portal.
I can get the expected result when I access the deltaLink.
Update:
I just noticed that you deleted the user directly. That's why you can only get removed user on user delta. If you remove the user from the group directly, you will get the removed user.