adobe photoshop - How do I apply anti-aliasing to the edges of a layer?

How do I apply anti-aliasing to the edges of a layer?

I have a layer that has rough edges. I want to apply anti-aliasing to it so it blends well against the background. What is the best way to do that in CS5?

Answer

Blurring will not give you a true anti-aliasing effect, and in some cases the blur will look even worse than the aliasing. If you want to do a really good job, keep reading…

Part 1: Create an Anti-Aliased Selection

Use one of the following methods. The first is hardest but gives you more control and can produce a much better result.

1. 
   

   Pen Tool – Select the Pen Tool and begin creating a path around the perimeter of the aliased shape. I will not explain how to use the Pen Tool. Check out the second option if you don't know how. Otherwise, draw the path, then go to the Paths panel, right-click on the path you created and choose ‘Make Selection...’ In the pop-up that appears, make sure that feathering is set to 0 and the Anti-Aliased checkbox is checked. Then click OK. A quicker way to make this selection if you are sure anti-aliasing is checked, is to Ctrl-Left-Click the path in the Paths panel.

   
   



2. 
   

   Magnetic Lasso Tool – Switch to the Magnetic Lasso Tool by holding the Lasso Tool button until all variants of the tool appear, and then select the Magnetic Lasso Tool. In the options bar, make sure Anti-alias is turned on and set the Radius, Contrast, and Frequency to 30px, 1%, and 100 respectively (play with different values if those don’t work for you). This tool works in a quirky way, you have to click the tool near the edge of the shape you are tracing, but don’t hold the mouse button down. Just move the mouse to trace the shape with the cursor as best you can, and then click on the beginning anchor-point to close the selection loop.

   
   


3. 
   

   Refine the Edge - This is the least accurate method, but can still produce a semi-decent anti-aliased selection quickly. Go to the Layers panel and Ctrl-Left-Click (Command-Left Click on Mac OS X) on the icon of the layer to be selected. This will create an aliased selection around your shape. Then choose Select > Refine Edge... via the main menu. In the Refine Edge dialog box, set Contrast to 40% and Smooth to 50 and set everything else to 0. You can also play with the Smooth and Contrast settings if those ones don't work for you.

   
   

Part 2: Contract, Invert, and Trim

Now that you have an anti-aliased selection in the same shape as the path, go to Select > Modify > Contract… via the main menu and enter 1 px in the dialog that comes up. Press OK in the dialog, and then click Ctrl+Shift+I on Windows or Cmd+Shift+I on Mac to invert the selection. Now press the Delete button on your keyboard to trim the aliased edge.

This should produce a true, anti-aliased outline of your object. It will be one pixel thinner than before, but in many cases that will be just fine.