A usual reason for not being able to delete a color is that it's used in a placed graphic. Use your separations preview to find it.
Is it a spot colour?
Yes it is a Spot Colour
Choose Window > Output > Separations Preview > choose View > Separations,
Turn off the CMYK swatches, zoom out and you should see the placed graphic select it and delete it and you can then delete the spot swatch to the trash in the swatches panel.
Frequently they are defined in a placed/linked graphic.
Create/Define a Preflight Profile
Color > Color Spaces and Modes Not Allowed > Spot color
Run the preflight, it will locate the graphic.
(Assuming the color is not defined in a Para/Char Style)
Thank you Derek & Daniel
It was a Graphic, not sure how it picked up that spot colour but I have found the problem with your help