The 6lb line will do funny things if you are catching fish or putting a strain on the line close to its rating. Kind of stretches it out and then it won't go back to shape and will curl. I would never troll with 6lb for this reason. Even the strain of a crank bait will do funny things to the line. I know you want a thin line thinking it will go deeper, but upgrade to a 10 or 12 lb test, then just let out more line to get your depth back on those diving crankbaits. The president reels come in both the baitcaster and spinning but would still say get rid of the 6lb and you will like it.
To do yourself a test. Take some of your 6lb, about 30foot, and tie it to a post. Then wrap a stick with some line on the other end and try stretching the line without breaking it. Just pull and let it go slack and see if it will curl. No barrel swivel will fix whats happening if your line is stretched.