My dcks are ages 15 months- 3 1/2 yo. Nobody seems to worry if their parent is last, they're all excited to see whose parent is picking up at the moment(I have great dcps).
Plus as the #s dwindle down they get more time alone with me, which they seem to appreciate at the end of a long busy active day with their friends. I do have to say that my 15 month old twins are the first to be picked up and they go all crazy gaga when mommy comes.
I agree with spedmommy, as long as the caregiver stays engaged with your 20 mo, it shouldn't be an issue.