Parameter Servers

https://d2l.ai/chapter_computational-performance/parameterserver.html

I’m a bit confused as to how ring synchronization becomes (n-1)/n. From what I understand, If you aggregate chunks you will need to have a full pass (n-1) on all gpus to aggregate a specific chunk. Could you please clarify?

I think you should know each GPU sends a chunk to next GPU ‘simultaneously.’
Thus, it suffices to consider the overhead of one GPU.
‘Each chunk is of size 1/n’
Each GPU sends the chunk n-1 times.
Thus, time is O((n-1)/n) w.r.t the number of GPUs.

I am a bit confused of the ring synchronization algorithm. After (n-1)/n time, each GPU should have only an aggregated chunk, not the full gradient. Is that right?