21 Replies Latest reply: Mar 16, 2010 7:05 AM by bsnyder175 RSS

    Why is PS allowing paint to spill outside of selections?

    bsnyder175 Community Member

      So, I have an image at high resolution. I want to cut an object out of the image, and then scale the image down to a smaller size. Now, when I command-click my cut out object, I receive a selection.

      However, when I try to paint within that selection, paint spills out onto pixels that are Not part of the selection. Does anyone know why this is happening? With the work I'm doing, it's imperative that the only part of my image that is being manipulated is my exact selection, and not a pixel over. But I can't understand why Photoshop is allowing a few pixels outside of my selection border to be painted on.

      Furthermore, if I were to do command-click my object for the selection and delete it out of the background image, it deletes the selection + a few extra pixels around the border. It's maddening. When I have a selection and manipulate the pixels within it, why is it also manipulating a small number of pixels surrounding the selection?

      Posted below is a video I made to demonstrate the problem. I start with a higher res image, cut out the picture, cut the image size in half, and then command-click my object to get its selection. Then if I delete that object out of the original image, it does 2 things: 1) deletes more than my selection and 2) doesn't delete what's IN the selection entirely. The edge of the selection in some spots become transparent instead of it being gone.

      Can anyone shed some light on this?

      http://www.youtube.com/watch?v=h58k7b_Wnts

        • 1. Re: Why is PS allowing paint to spill outside of selections?
          Paul Cracknell CommunityMVP

          Hi there,

           

          I took a look at your video, to me it looks like the anti-alas edge. Try turning off the Anti-Alas check box next to the feather amount in your Options bar.

           

          Paul

          • 2. Re: Why is PS allowing paint to spill outside of selections?
            bsnyder175 Community Member

            Thanks for the quick reply, Paul.  I saw the option for anti-alias in marquee, lasso, and magic wand tools.  In the marquee tool, it's grayed out (although it's unchecked anyway).  I'm not sure I understand your solution.  After I command-click my layer to obtain its selection, I should uncheck the anti-alias box?  In which tool?

            • 3. Re: Why is PS allowing paint to spill outside of selections?
              Paul Cracknell CommunityMVP

              The Anti-alas check should be off before you create your selection. Think of Anti-alas as applying a small feather when a selection is created.

              • 4. Re: Why is PS allowing paint to spill outside of selections?
                c.pfaffenbichler Community Member

                Additionally I suspect selecting for painting is unnecessary here as one could add a Layer and Clipping Mask that to the existing cropped Layer (alt-click on the line between the Layer symbols in the Layers Panel).

                • 5. Re: Why is PS allowing paint to spill outside of selections?
                  Reynolds (Mark) Community Member
                    In the marquee tool, it's grayed out

                  Yes.  the straight marquee tools do not have anti-aliasing as an option.

                   

                  Its your downsizing that is making the edge of your selection no longer straight. In your preferences you can choose Nearest Neighbor for your interpolation method (first pane of preferences) . If you are going to exactly 50% then this will be Ok. It won't be OK for most resampling though, so best not to leave it that way.

                  • 6. Re: Why is PS allowing paint to spill outside of selections?
                    bsnyder175 Community Member

                    Thanks all for the information.  You've all been very helpful.  So, it seems that the issue is when I downsize images.  It's going to be different almost everytime - 50% was just an example.  I guess the only thing I still don't understand is why my selections are able to be clone stamped or painted beyond their borders.  That's really what I want to avoid.  Maybe it's been explained already in this thread and I just didn't catch it.

                     

                    It's almost like the selection is actually larger than that "marching ants" by a few pixels.  And I'm pretty sure this issue is able to happen at the images full resolution.  It seems most common on slanted edges.

                     

                    This is more of what I'm Trying to do:

                     

                    cut out selections of an image that are side by side.  When I downsize, some of the pixels between my side by side selections are lost.  So I clone stamp the edges back; however, the cloned pixels spill Over the selection and onto the object next to what I'm cloning. It's not noticable in photoshop, but when it's brought over into another application, the overlapping pixels are extremely obvious.  This theoretically should be solved if photoshop would stay true and only let me clone stamp within my selection, instead of allow pixels to spill outside of the selection (creating overlapping edges).

                     

                    If this is all confusing, I'll try to put up another video soon to better demonstrate what I'm doing.

                     

                    Thanks again very much for the responsiveness to my issue.

                    • 7. Re: Why is PS allowing paint to spill outside of selections?
                      Reynolds (Mark) Community Member

                      bsnyder175 - when working at really low resolutions like you are, there will always be a certain error factor when it comes to accuracy. May seem like an obvious point, but this is because pixels are… square or rectangular. Any uncertainty whilst working within this rigid grid structure has to be interpolated in some way. If your mouse position is even partly out when clicking with the lasso, photoshop has to assume which precise pixel area you meant.

                       

                      To force photoshop to create blocked pixel shapes you need to use anti-aliasing options, or Nearest Neighbor as the resampling method. Work at higher resolutions.

                       

                      There are various solutions while working with selections like the Refine Edge tool, and the Expand Contract commands.

                      • 8. Re: Why is PS allowing paint to spill outside of selections?
                        carson.jones Community Member

                        I scanned through the post quickly and someone may have noted this already. 

                         

                        When Command / Clicking to create a selection, PS will remember your last Feathering setting.  Go to Select / Modify / Feather and make certain the number is set to zero.

                        • 9. Re: Why is PS allowing paint to spill outside of selections?
                          bsnyder175 Community Member

                          My feathering option for some reason was required to be at least .2.

                           

                          This video should be a much better demonstration of my problem.  If you wouldn't mind, could you please take a look and give me your thoughts?

                           

                          http://www.youtube.com/watch?v=I44sTR6Nj-A

                          • 10. Re: Why is PS allowing paint to spill outside of selections?
                            carson.jones Community Member

                            Very strange.  The video was very helpful in trying to understand your problem.  Have you tried converting the group / layers to a smart object and then resizing?  I'd be curious to know if this still occurs.

                             

                            As to why you're forced to use a feather setting of 2 I'm at a loss.  Usually choosing Select • Modify • Feather from the main menu allows you to set the feathering to zero.

                             

                            A few questions / thoughts...

                             

                            - Are you cloning / stamping your selections to fill pixels that were solid before you resized?  I'm trying to understand why you're having to stamp the layers at all if they were fine at the larger size.

                             

                            - If you're loosing your edge to transparency after sizing, perhaps you can try this.  Instead of activating a selection and then stamping in the newly semi-transparent edges, simply duplicate the layer on top of itself until the transparency fills back in.  This should make your semi-transparent edges solid again without going beyond the object edges.

                             

                            - I noticed in the video that you had three layer elements stacked to create one single piece.  Are you able to merge these parts (layers) before scaling or is there a reason you need them as separate layers?

                             

                             

                            I'll stay on top of this one - interesting problem!

                            • 11. Re: Why is PS allowing paint to spill outside of selections?
                              carson.jones Community Member

                              I had another thought...

                               

                              Prior to scaling the image down, lock the transparency on each of the layers.  Perhaps locking the layer transparency before scaling down will prevent the edge issue.

                               

                               

                              _____________________

                               

                               

                              Follow up...

                               

                              I should have watched your first video from the outset!!!

                               

                              Try this...

                               

                              Instead of using a marquee tool to make your initial selection on the high resolution image use the pen tool to create a path around your object.  This should be very straight forward given that it's a rectangular frame you're selecting.  Convert the working path to a saved path by double clicking on it in the Paths Panel - name the path appropriately (e.g. Frame).

                               

                              Next command click on the path in the Paths Panel making it an active selection.  Proceed to duplicate the artwork to its own layer.  Choose Deselect (Command + D on the Mac).

                               

                              Scale the image.

                               

                              Once scaled, command click on your saved path in the Paths Panel and proceed as you had done in the video.  I think this should resolve your problem.

                               

                               

                              ______________________

                               

                              Also, is there a reason you're waiting until you've scaled down to delete the selection from the background layer?  This might not be an issue if it's done prior to scaling down.

                              • 12. Re: Why is PS allowing paint to spill outside of selections?
                                bsnyder175 Community Member
                                - Are you cloning / stamping your selections to fill pixels that were solid before you resized?  I'm trying to understand why you're having to stamp the layers at all if they were fine at the larger size.

                                no.  If I'm zoomed All the way in, it shows my pixels sitting side by side perfectly.  However, if I zoom out, even from full resolution, it begins to show the missing edge pixels.  And then of course, when I resize down, they're completely obvious.

                                 

                                - If you're loosing your edge to transparency after sizing, perhaps you can try this.  Instead of activating a selection and then stamping in the newly semi-transparent edges, simply duplicate the layer on top of itself until the transparency fills back in.  This should make your semi-transparent edges solid again without going beyond the object edges.

                                I'll be trying this option tomorrow, thanks.

                                 

                                - I noticed in the video that you had three layer elements stacked to create one single piece.  Are you able to merge these parts (layers) before scaling or is there a reason you need them as separate layers?

                                It's imperative that the layers stay separated, as they're saved down as single images and then put back together as a whole in another interactive application.

                                 

                                Thanks a whole bunch for sticking with me on this problem.  It's a very welcoming feeling to know I'm not fighting this issue alone.

                                • 13. Re: Why is PS allowing paint to spill outside of selections?
                                  bsnyder175 Community Member
                                  Prior to scaling the image down, lock the transparency on each of the layers.  Perhaps locking the layer transparency before scaling down will prevent the edge issue.

                                  Great thought, but that didn't make a difference.

                                   

                                  Instead of using a marquee tool to make your initial selection on the high resolution image use the pen tool to create a path around your object.  This should be very straight forward given that it's a rectangular frame you're selecting.  Convert the working path to a saved path by double clicking on it in the Paths Panel - name the path appropriately (e.g. Frame).

                                   

                                  Next command click on the path in the Paths Panel making it an active selection.  Proceed to duplicate the artwork to its own layer.  Choose Deselect (Command + D on the Mac).

                                   

                                  Scale the image.

                                   

                                  Once scaled, command click on your saved path in the Paths Panel and proceed as you had done in the video.  I think this should resolve your problem.

                                  Interesting thought.  I'll give that a shot as well tomorrow.

                                   

                                  Also, is there a reason you're waiting until you've scaled down to delete the selection from the background layer?  This might not be an issue if it's done prior to scaling down.

                                  I'm not sure I understand your question.  What do you mean by "deleting the selection?"

                                  • 14. Re: Why is PS allowing paint to spill outside of selections?
                                    bsnyder175 Community Member

                                    One more thing.  I'm not sure if you noticed or not in the last video, but as soon as I scale down, I get the transparent edges.  Clone stamping only enhances those "spilled over" pixels.  They're there as soon as I scale down though.

                                    • 15. Re: Why is PS allowing paint to spill outside of selections?
                                      carson.jones Community Member

                                      bsnyder175 wrote:

                                       

                                      - Are you cloning / stamping your selections to fill pixels that were solid before you resized?  I'm trying to understand why you're having to stamp the layers at all if they were fine at the larger size.

                                      no.  If I'm zoomed All the way in, it shows my pixels sitting side by side perfectly.  However, if I zoom out, even from full resolution, it begins to show the missing edge pixels.  And then of course, when I resize down, they're completely obvious.

                                       

                                      To clarify... when you're zoomed in at 100% you're not seeing missing edge pixels but when you zoom out you are seeing them.  If this is the case it could be a notorious previewing problem that PS has had for years and continues to have.  Whenever you are viewing your files on-screen and are not zoomed in to 100% of actual, there is a chance that you are seeing a misrepresentation of the image.  Pixels tend to disappear and reappear depending on your zoom level.  Are you seeing this in the lower resolution files when you're zoomed in at 100%?

                                      • 16. Re: Why is PS allowing paint to spill outside of selections?
                                        bsnyder175 Community Member
                                        To clarify... when you're zoomed in at 100% you're not seeing missing edge pixels but when you zoom out you are seeing them.  If this is the case it could be a notorious previewing problem that PS has had for years and continues to have.  Whenever you are viewing your files on-screen and are not zoomed in to 100% of actual, there is a chance that you are seeing a misrepresentation of the image.  Pixels tend to disappear and reappear depending on your zoom level.  Are you seeing this in the lower resolution files when you're zoomed in at 100%?

                                        Correct.

                                         

                                        Full resolusion:

                                         

                                        all the way zoomed in, it's perfect.  Just how I'd want it to be.

                                        100% of image size, they're still usually ok.

                                        zooming out further (to the approximate size of my image, after it would be resized), I begin seeing the edge transparency

                                         

                                        End resolution:

                                         

                                        in any zoom level (100%, zoomed in, or zoomed out), the pixels are just plain gone and very easy to see.

                                        • 17. Re: Why is PS allowing paint to spill outside of selections?
                                          carson.jones Community Member

                                          Did a bit of testing and think I've resolved your issue...

                                           

                                          Your problem is likely being caused by the image scaling setting.  By default it's probably set to bicubic smoother.  Change it to Nearest Neighbor and your issue should be resolved.  I have attached a screen capture of the dialogue box.

                                           

                                          Picture 13.jpg

                                          • 18. Re: Why is PS allowing paint to spill outside of selections?
                                            bsnyder175 Community Member
                                            Your problem is likely being caused by the image scaling setting.  By default it's probably set to bicubic smoother.  Change it to Nearest Neighbor and your issue should be resolved.  I have attached a screen capture of the dialogue box.

                                             

                                            That definitely solves that issue.  However (and you're going to hate  me for saying this), I'm of course, left with jagged edges around the  rest of the "object."  BUT...I thought I could use the best of both  worlds.  I combined both 800x600 (bicubic and nearest neighbor) resizes.  I used  the bicubic version for the majority of the object, and overlayed the  nearest neighbor object (the layers) on top of the bicubic folder.  Then I cut  out everything except where edges met eachother.  So, the only place it  gives jagged edges (which aren't that apparent) are where edges meet -  but that's Far better than having them overlap.  It might sound confusing, but I think it will work!  I'll submit this to the  boss, with your props, and see what he thinks.

                                             

                                            Thanks  much, Carson.  You're one of a kind.

                                            • 19. Re: Why is PS allowing paint to spill outside of selections?
                                              Reynolds (Mark) Community Member

                                              So when I said back in post 5, more than a week ago "you can choose Nearest Neighbor for your interpolation method" - you were not listening obviously

                                              • 20. Re: Why is PS allowing paint to spill outside of selections?
                                                carson.jones Community Member

                                                Reynolds (Mark) wrote:

                                                 

                                                So when I said back in post 5, more than a week ago "you can choose Nearest Neighbor for your interpolation method" - you were not listening obviously

                                                 

                                                 

                                                 

                                                Just saw that now. 

                                                • 21. Re: Why is PS allowing paint to spill outside of selections?
                                                  bsnyder175 Community Member
                                                  So when I said back in post 5, more than a week ago "you can choose Nearest Neighbor for your interpolation method" - you were not listening obviously

                                                  Incorrect.  I just never thought about combining them until Carson mentioned them again.