Operating system deployment slowness at remote sites.
While checking the SMSTS.log and DataTransferService.log logs on TEST001 (Testing machine) (this one took 9 hours to finish), I found out that the packages and applications were getting downloaded from One of the Primary site (PRI01). We wanted all the packages and applications to be downloaded from Remote distribution point (RDP01) as this server is local on remote network (RN01) and has all the required packages/applications. I looked at the boundaries on PRI02 site (RDP01 belongs to PRI02) and everything was correct. I tried to redistribute packages, delete and re-add the boundary and boundary group for Remote office, but none of these fixed the issue. After struggling for a bit, I decided to check the boundary on PRI01 site.
When I connected to PRI01 site, I realized that there was a typo for boundary IN CRP - 1. Below is the snapshot of the IN CRP - 1 boundary that I found.
As you can see here, the starting IP is 10.193.8.1 and ending IP is 188.8.131.52. This includes all the IP addresses between these two IP addresses, which includes all of our remote sites as they have IP addresses like 10.202/204/205/210 etc. Because of this conflict, the client was thinking that although it belongs to PRI02 site, its currently roaming in PRI01 site and hence was using PRI01 distribution point. After I changed the ending IP address to 10.192.11.254, the client in RN01 started downloading packages/applications from RDP01 and TS execution time went down from 9 hours to just under 2 hours.
So going forward, please pay special attention to details like this, any miss-configuration can lead to issues at some other office location.