Wow, Kevin. That sucks.
I think you have a pretty clear business case to talk to Adobe at. Since there was some clear financial cost here, and clearly some budget available, its probably appropriate to engage Adobe support and your sales folks, if you have any. Especially since I suspect this is going to come down to an intermitant bug of some sort.
I'm sure you aware that printing to Postscript and then making a PDF is not the normal workflow for PDFs in InDesign. The normal workflow is to Export to PDF from InDesign. Can you tell us why you were printing to Postscript?
Do you have the original Postscript files that InDesign produced? While, as you say, it is pretty much impossible for Acrobat Distiller to be at fault here, it is quite possible that inspection of the postscript file it processed can give you some important clues. Since Postscript is an ascii text-based language, you can open the PS file in a text editor and inspect it near some of the footnotes.
It is also plausible to do some level of forensic analysis on the PDF files, but you'll be more dependent on complex tools and will be also one cooking step further along in the process.
I assume you are using InDesign's Book feature with multiple INDD files (44 of them?).
I would try printing to postscript with a wide variety of different settings in the print dialog, trying to see if you can replicate the problem. I would have to guess that it is something entirely unrelated to footnotes, if it is triggered by a setting, since there really isn't a setting to "scramble and drop half the footnotes."
Have you tried watching over the shoulder of the individual human person who ran the export that failed, watching them from start to finish as they start inDesign, open the book, and export the postscript? Perhaps they are doing something non-intuitive that you would never try (or that doesn't even look wrong but just has
unintended consequences!) that triggers this bizarre behavior.
Of course an analagous question is whether that individual can reproduce the problem. Clearly you cannot, but maybe they can?
None of this is very helpful I fear. Just my off-the-cuff thoughts. Good luck.
Even the guy who made the PDFs can't replicate this issue–and he has been doing this a long time. I doubt Adobe would be able to replicate it either. We delete the .ps file after distilling so, no, we don't have those to examine.
RE: "normal" PDF workflow
While exporting is probably the most-used method of making PDF these days, printing postscript and Distilling is perfectly valid as a workflow. It just works–and the potential for certain issues are taken off the table.
The book had 44 chapters, but this was just one InDesign file.
I figure some type of flakiness with numbering and sectioning caused the issue. It manifested itself as though someone had gone in and changed section start locations, but I checked multiple times–they are correct. Also, some footnotes were just gone. For example, in one chapter, the footnotes were running along 33, 34, 35, 36, 43, 44, 45, etc… Gone. In another instance, the footnotes are humming along 23, 24, 25, then 2. Next chapter starts, the first footnote is 3, then 4, 5, etc… but the section start is in the right location, on the page where footnote 3 begins the sequence for the chapter.
No, suggestions, but if you report the problem to Adobe and make a big stink of it (do you have any pieces of shredded book to mail to them?), perhaps they'll put two and two together with someone else's bug report and find the cause or manage to put in some checks to prevent it from happening again, even if they don't know the cause.