if you're noticing that the plastic is sliding up the hook, or the fish somehow manage to steal it off the hook, then i'd use some superglue.
as long as you let it dry first (which shouldn't take long) i don't think there'll be any scent or taste issues.