One possible model of the universe is a low-entropy "bounce" in the middle (the big bang), with entropy increasing in both time directions away from it (and therefore observers on either side of it experiencing the big bang as being in the "past"). Then the only part that remains to be explained is why there's this single point of low entropy, and that's kind of a "why is there something rather than nothing?" question.