Marching Ants Emulate Edge of Selection - Rectangular Marquee Tool

[osbthecreator]

Hi all. I'm having an issue with the rectangular marquee tool. I have been using it all day without incident to select and fill in (with the paint brush) sections of a graphic so as to achieve crisp straight lines. Now suddenly when I select a section I notice that the marching ants, rather than adhering to the straight line of the rectangular shape, emulate the edge of my selection (pic below). Is there any reason why my ants might be marching a drunken line? When I initially make the rectangle they are marching in straight lines; it's only when I use transform selection so as to position the rectangle over the graphic that they begin to emulate the edges of the graphic. Any suggestions would very much be appreciated. Thanks!

[davescm]

Is the canvas rotated slightly? To check, press R  and check the angle in the options bar

Dave

[osbthecreator]

Hi there, thanks for the reply and suggestion! I checked and my canvas is not rotated even slightly. I wonder if you wouldn't mind elaborating on your follow up post about lines being shown at a very slight angle being stepped pixel by pixel? I admit I am quite new to Photoshop and so I am unfamiliar with this terminology. Thanks!

[c.pfaffenbichler]

When I initially make the rectangle they are marching in straight lines; it's only when I use transform selection so as to position the rectangle over the graphic that they begin to emulate the edges of the graphic.

Maybe I misunderstand but when you transform the rectangular Selection what do you expect but having a transformed Selection as a result?

And Selections being pixel-based anti-aliasing on originally »binary« edges should not be unexpected (in accordance with the Interpolation settings).

Might you not be better served using Paths and Vector Masks?

[davescm]

I just notice that the original poster selection had been transformed !.   If that is the case the only way Photoshop can show lines at a very slight angle is to step them pixel by pixel. So your post is absolutely correct and should get the "correct answer"

Dave

[osbthecreator]

Hi there, thanks for the reply and suggestion! My only understanding of using the transform selection option with regards to the rectangular marquee is that it is what enables me to move and resize the rectangle without displacing parts of the graphic beneath it and so it was with this intention that I was using transform selection—simply to freely move the rectangle about. Am I to understand that the transform selection option has other implications? And as I said, I had been using the rectangular marquee and using transform selection (to move about and resize the rectangle) throughout the day without incident in that the lines of the rectangle remained straight even when in contact with the similarly wavy lines of my graphic (the green line in my picture being an example of one). All that said, I would like to explore your suggestion of using Paths and Vector Masks but I'm really quite new to Photoshop and so I don't know exactly what you mean—would you mind elaborating? Thanks!

[D Fosse]

Frankly I don't understand what you have an issue with - and apparently others are similarly confused.

Is it the fact that you first select the whole canvas - but then as you start to do something, the selection gets restricted to the actual content of the layer? That's normal if you have partial pixel content but the rest of the layer is transparent (empty).

If that's not it, I can at least say that everything you describe or show here seems perfectly normal.

[osbthecreator]

Hi there, thanks for your input. Allow me to clarify.

First, I draw a rectangle. It is as it should be in that the edges are perfectly straight.

Then, I choose Transform Selection because I wish to use the rectangle to enclose the vertical line on my canvas (pictured green) and so I need to flip and resize it but I do not wish to affect the design itself with the selection; rather I wish to use the rectangular selection to fill in the pictured line with the paint brush so that it is perfectly straight. It is my intention to use the edge of the rectangle as a guide and to contain the paint to the specified area.

When I rotate my rectangle and move it over the green vertical line of my design, the edges of the rectangle are no longer perfectly straight, but rather the marching ants seem to be emulating the slight waves of my design.

I am not selecting the whole canvas, and it is not my intention to do so.

Is my issue a bit clearer now?

[davescm]

HI

When you rotate a rectangle the sides are no longer vertical/horizontal. They are now angled (unless you rotate it exactly 90 degrees).

With slight angles Photoshop will display the line of marching ants in steps. It has too as it is restricted to using either column A of pixels or column B next to it. There is no in between. That is why the marching ants appear to step if they are not perfectly vertical or horizontal.

Dave

[osbthecreator]

Thanks so much for clarifying, this makes a lot of sense.

[osbthecreator]

Perhaps I need to ask a different question. If what I wish to do is fill in the section circled in red (pictured below) with the paint brush or clone stamp and I need to ensure that the paint is contained and the edge that it creates is perfectly straight, what is the best way to do it? Thanks in advance for any and all suggestions.