I am running the Clam plate with a Mora auger (twin blade) also. I have both a 6" and an 8". With the 6, I ran it straight off the Clam plate as it came out of the box with no wobble at all. This year I stepped up to the 8" and added on the Clam gear reduction drive as well as an extension to the auger shaft. Slower cut, but bigger hole and longer run time on the batteries. I am using the 1000+ inch pound Ridgid drill and 4amp hour batteries. Easily cuts 400+ inches with the 8" auger on a charge. Still no wobble on my end.
I also tried an Eskimo auger in the past. Not sure the name but I believe it was an 8" and it had the scoop style blades. Problem though was it had no piercer point to help keep it centered, so it would bounce around dangerously under power. Total fail.
Have you tried running your setup at different speeds? Mine is always on the slower speed higher torque setting on my drill. Maybe pull the drive axle off of your plate and make sure it is straight, as well as checking for any chance of a bent auger shaft. If it is bent, Clam has replacements, but if you have to replace it, consider stepping up to the gear reduction unit. It will allow up to a 10" auger no problem should you decide to upgrade over time. And if you want to stay with a 6" you could probably run it in the higher gear on your drill for blazing fast cuts.