Your wiring seems okay. But a bit messy. I recommend not splicing so many wires in random spots. And do not leave so much exposed wires.
As for hissing. That it pretty normal with pwm audio. You will likely always have some hissing. Also just to confirm. You have the proper text added to the config.txt?
That little amp also has a small screw pot on it (above a+, use a screw driver to adjust) I recommend maxing the volume in retropie. (Press start and find volume in there) then adjust the screw pot on the amp up or down to see if you get some sound.
For reference this is my audio diagram. Using an original headphone jack. I wrap the signal wire with the ground wiring. It seems to help with noise.
It's also a little hard to see what going on with the headphones. But make sure you check here for the pinout of your jack. You can always connect the speaker direct to the amp to ensure it works before connecting the headphones.
https://sudomod.com/forum/viewtopic.php?t=701