What the individual Spirit chooses to do in that "off-duty" time is the topic of a long discussion between the GM and the summoning character's player.
It might be influenced by the individual Summoner's beliefs/idiosyncracies/PQs/NQs, or the type/classification of spirit.