From what video camera was the clip captured?
Welcome to the forums.
Ignore the clip on the timeline. Delete the clip from your Project Panel and it will also be removed from all timelines.
With only one drawback, you also lose the video part. Can't you unlink audio and video and then delete the audio part, or has the crash already occurred before that. If the latter is the case, my guess is that you are using non-editable material as your source with the wrong codec. So give details and start with answering my earlier question.
Not sure what method you are using to remove the Audio Clip, and some potentially important questions have been asked. While you're answering those, try one thing:
Alt-click on the Audio Clip in the Timeline. It *should* now be Selected, but without the accompanying Video. If you have not crashed yet, hit Del now, and X your fingers. If you do crash at that point, we'll definitely need the answers.