Could you please check in the Preferences of Lightroom, under External Editing, the option " Stack with Original " should be checked.
Also the screen shots are not showing, could you please attach the screen shot.
stack with original is on, I will add the screenshots tomorrow.
Starting point , notice the folder from here I changed the photo in PS
After turning the photo 90 degens I saved the it in PS, NOT saved as ....
The photo was not returned to the original folder but one level up.See the folder structuren at the left.
My settings in LR >
As mentioned, working with other software gives the same result. Result is not stored in the original folder.