If you’re referring to the frame edge display that is determined by the color you’ve assigned to the layer.
That can be changed in the layers properties.
As described, I am talking about the edge of the image, not the frame
I think you mean the image bounding box when selected with the direct select tool. That's also controlled by the layer color, but I can't tell you the algorithm used.
I'm pretty sure the only way to change that color is to change the layer's main color.
I'm the kind of geek who found this interesting enough to check into it some, and it appears the Adobe Engineers hard-wired each layer color with a "contrasting" direct-selected image edge-color. Once you know how they are paired, you can always set particular layer color for your image layer, thereby enjoying your choice of image-edge color. Here are the first 12 color pairings in order of default new-layer creation:
Layer Color > Image Edge Color
Lt Blue > Brown
Red > Cyan
Green > Purple
Blue > Yellow
Magenta > Dark Green
Cyan > Red
Orange > Blue
Dark Green > Magenta
Teal > Orange
Tan > Black
Brown > Teal
Violet > Green
I think you are correct that there is no way to change the image edge color or as Peter correctly identified it as "image bounding box". But you have helped by identifying the color combinations of the layers. I will keep away from the layer colors that give me a light bounding box color. Unfortunately black is a layer color I like to use and that seems to give the lightest bounding box color. It's very rare that I create a file that has 12 layers so I will be printing out your list and referencing it often.