Rather than using the alpha channel as a clipping mask, you might want to try removing the background and saving a transparent file. I can't promise it will work better (another user reported a very similar problem earlier this week which remains unresolved, as far as I know). TIFF is less likely to show a black halo, though, in the PDF than a transparent PSD.
I did use TIF as the format, which is what everything I read was recommending. The only thing I can think of that may be the issue, although I would have to test it out, is that some of the photos are RGB and I'm exporting a CMYK PDF for print (or vice versa when I export an RGB PDF for web viewing, as some images are CMYK), but I am choosing the preference to convert images to the proper colorspace, so I was under the ipression that would fix any color issues. Besides that, my file is conforming to all of the standard practices that I've used for years and never had any problems with (300 dpi, never going above 100% scale, using tif format, etc.).
I've used quark primarily in the past, but prefer InDesign now that I've started using it, due to the way it's so similar to Photoshop/Illustrator and with all of the nice features they have built in to assist with building quick layouts. I've just never seen this issue with a pdf before and the file looks perfect except for the strange looking edges. It's only on masked images, so it must be soemething I'm doing...just can't figure out what it is. I will try the transparent BG on a psd and see what happens, but that seems like it should be a more problematic method then a TIF file would be. It's worth a shot.
Changing the image to a psd with transparent background did not change anything for the most part. The mask did look a bit different when exported (not as clean to the image as it was in the TIF), but it still had the black line around the image.
One strange thing I am noticing, is that the black line is only showing up in areas that are a hard line on the mask, i.e. black going to white. Where I have gradients in hair and things of that nature, the mask looks as expected in the pdf.
I thought maybe it had to do with the fact that I have a vector path on the image, as well as the alpha channel, but that did not seem to change anything when I removed the vector. It looks perfectly fine when I zoom in to 100% or greater, seems to only appear on the pdf at anything less than 100%.
An interesting note...I just tested the PDF in Preview on Mac and it looks fine. No black line at all and the mask looks perfect. Maybe it's just my Adobe Acrobat doing it?
I was suggesting transparent tiff, not .psd, instead of the alpha channel mask. I'd also suggest export to a setting that doesn't flatten transparency (though that may make things look bad in Preview). I don't have Acrobat X, so I can't say if this is an Acrobat or ID problem as far as version. I can say that the other user was having similar issues with ID CS5.5 and Acrobat X, but not, if I recall, with CS4.
I don't normally mix color modes on jobs that need flattening, partly because I find it very confusing (and I Rarely use transparency on a file I'm going to flatten). My tendency would be to use the blend space that corresponds to the background, though.