It is uncovered that reducing such pairwise class confusion leads to significant transfer gains, and a general loss function is proposed: Minimum Class Confusion (MCC), which can be characterized as a non-adversarial DA method without explicitly deploying domain alignment, enjoying faster convergence speed.