I finally figured it out. I added back the missing alias. Then I just deleted the alias and the site listing in manage > sites is now gone.
I know it makes no sense. One place the extra site still shows is in the dropdown above the title on the persona bars settings > site settings but I'm just going to ignore it and hope it goes away.