"Pushed back" refers not to moved back in time, as if to an earlier time; it refers to something being moved back in abstract space. Â In this scenario, whatever is pushed back will be encountered further along, such as a politician sooner meets those in the crowd or a line who push forward (to the front), and those who are pushed back he/she meets later. Â Â
It can also refer to time, but in that case I take it to mean pushed to a LATER time, not earlier. Â "Sorry, I'm not going to make it; we'll have to push back our meeting."