This paper proposes substructure-level matching for domain adaptation (SSDA) to better utilize the locality information of activity data for accurate and efficient knowledge transfer and proposes an optimal transport-based implementation, Substructural Optimal Transport (SOT), for cross-domain HAR.