I finally found and fixed the issue! So after lots of testing on the board without the raspberry pi, I finally tested with it plugged in and found a short in two of my header pins. I resoldered the header pins back onto the pi and everything works again. I feel like an idiot for not checking that a long time ago. I ended up replacing the ATMega, and resoldering quite a bit of the board, only to find out it was my sloppiness in soldering the pi header pins.

Thank for the help Ralph and Bryan, it was much appreciated. And as an added bonus, I now understand the circuit much better than I would have before.