If you enable Anonymous Authentication, the user will not get authenticated and you will never know who has invoked the process( if you have set the Run As option to "System").You may ask the user to supply UserId and Password (in base 64 format) and then authenticate within the process using the AuthenticationManager API.Nith
1) Could I configure jboss to issue a redirect to a custom html page when a 401 exception accurs?
Ans : Yes you can handle it in front end in ajax by having a seperate exception handler in the server side which redirects the page when 401 exception is thrown from adobe
2) Could I configure jboss to return a 403 or 500 error when there is a 'not Authorized' exception, instead of the 401?
3) If I set "Require callers to authenticate:No" and 'Run as System" for the service is there any way I could then check within my process to see if the current user is actually authenticated?
No you can not since you set "Require callers to authenticate:No" . You can track your session in adobe side only when the user authentication is set to yes