it would be way easier to just make a keyframe of each location and have after effects always trying to put the object back where it came from.
Would it? Unless you do it for hundreds of keyframes, cobbling together the expression would take longer than just hand-animating it. The rest doesn't matter. Whether it's 2D or 3D or just 1D - you still need to calculate the timing and positions. That's as trivial as running through the same interpolator as often as needed for each axis. The basic drill is to find out between which keyframes you are using nearestKey(), calculate the actual duration and then feed that into a linear() interpolator to blend between the positions, where the driver is something like Math.pow(time,2), i.e. a square falloff as gravity would do. And here comes the stinker: Since AE is not able to store persistent variables for expressions and you thus never know where the object is stuck, it would be one big hog of a loop to calculate this over and over.
You are also not making any determination how the motion force to counteract the gravity is actually applied. Is it some imaginary "force" impulse? A fake moving into the position with a fixed speed or easing duration? Does the object need to respond to other effects like sideways acceleration forces? Need the curves to look "physical" or can the just be linear paths? You see, it's much more complicated. A simple gravity bounce entirely based on an expression would be easy enough, but combining physics with keyframe control is a whole different one. If you want an idea what you would be getting into, study the "Trigger animation with markers" and "Elastic connection" examples on Dan Ebberts' page:
So for what it's worth, if you can do it by hand, do it by hand. Otherwise we're talking some serious code mangling.