Occasionally losing network connection depending how the network cable is inserted

Any tips how to repair or work around this?

 

The lights go off if I accidentally touch the cable the wrong way. Trying to jam/stick it in more securely doesn't seem to help.

Done