I think you need to slow the motor down even move. My guess is that at the correct speed you will get the motor and the swing to get into a repeatable "rhythm", You might also have to get the swing going by hand at first.
You need to slow the motor down. If you can not do it electrically use a small pulley on the motor and a bigger pulley and hook you pushrod to swing there. Like a 2 inch on the motor and an 8 inch on a shaft leaving one side open to attach you push rod. Also use any of the suggested methods to stiffen the rope all the way to the seat... good luck
My Swinging skeleton would do the same thing years ago. Like others said, the problem was the pivot point where your PVC meets the ropes. The PVC wants to swing, but the rope linkage won't allow that motion to be transferred to the swing itself.
I solved the problem by removing those linkages.
Try attaching your swing ropes about 4 or 5 inches up onto the BACK of those PVC pieces. Use a screw to secure the rope end to the back of the PVC. Then take some bailing wire and wrap it around the rope and PVC a couple of inches below the rope end. Then another wire right at the bottom of the PVC.
Try that and see if it improves it.
Here's a pic of mine:
Here's mine in action: (Old video-crappy looking.)
Without having the ropes rigid you might have a hard time getting it to work how you have it setup. Possibly try to use the idea in the video, describing how changing the length of the pendulum can keep the swing going. .