Are you deleting it from a local HD or a "remote hard drive"?
If Lightroom is referencing the photo on a local HD, it will not delete it on the "remote hard drive".
Exactly what do you mean by "remote hard drive" anyway?
I have a LaCie 3TB remote (usb connected) drive where I store all my photo files worked on in LR.
Do you have write access to this remote drive?
Are you Removing>Deleting From Disk the DNG file? Or is this a TIF, PSD, JPG you created, Exported, from the DNG you are Deleting from disk?
Deleting a JPG, TIF or PSD file does not remove, Delete, the corresponding, Original, DNG file.