I too am having this issue of being unable to completely delete groups. I'm having to renumber them out of the way to re-use the numbers I want.

It's not a massive issue really but my OCD kicks in and I want to number them in order!!!

Cheers.