Check your footage interpretation. That aside, AE doesn't care about file sizes. If there is an issue it has more to do with the internal structure and possible damage to some streams. A quick transcode in tools like Handbrake can fix these things most of the time without any quality loss.
What do you suggest I do with my footage interpretation? I'm kinda an AE noob and don't know what to do with it other than set it to the same frame rate as the footage.
Also, even after transcoding in Handbrake I had the exact same issue.
What did you transcode it to with Handbrake?
How is AE interpreting the file when you import it?
When initially importing the footage I get the message: "After effects error: overflow converting ratio denominators.
(17 :: 18)"
Then I chose "Conform to frame rate: 60"
The only options that I saw for transcoding in Handbrake were .mkv and .mp4 so I transcoded it to .mp4 with the h264 codec. I got a file about 1/3rd of the size but with the exact same issues as before.
You don't want to transcode to another lossy codec! Also, it's a long GOP codec which is the whole problem in the first place. Try using the Adobe Media Encoder (or Handbrake) to make something like QuickTime with ProRes codec, DNxHD codec, Cineform codec, Photo-JPEG codec, or the PNG codec
Are you still facing this problem? If not, let us know how you solved it. If so, please let us know so we can assist you further.