Theoretically speaking, the Pope can wear whatever he wants. But, in both public and private wear, popes often consider themselves bound to the precedents set by their predecessors. For example, popes wear white cassocks for ordinary wear because Pope St. Pius V set the precedent for popes wearing white by choosing to wear his Dominican habit while pope. Another example: In a recent interview of Pope Benedict XVI by journalist Peter Seewald, published as the book
Light of the World, Seewald asked Pope Benedict if he wears casual clothes such as sweaters during his private time. No, the Pope answered, he was told by one of Bl. John Paul II’s personal secretaries that “The Pope [referring to Bl. John Paul] always wore a cassock and so must you.” That’s not strictly true. Pope Benedict could wear a sweater in his private time if he wants to, and he himself has not always followed all of Bl. John Paul’s personal customs, but his acceptance of this “mandate” from someone who loved and served his predecessor is an indicator of Pope Benedict’s personal humility.
This attitude of the popes on this issue is summed up by an interesting anecdote from the life of Pope St. Pius X. St. Pius X was a secular Franciscan, and it was a rule in those days for secular Franciscans to wear a habit given them by the Franciscan order. When he was elected pope though, St. Pius found wearing the habit under his papal clothing to be extremely uncomfortable, so he called in the Minister General of the Franciscan order and asked for a dispensation from wearing the habit. The Minister General was startled, and said, “Your Holiness, you’re the Pope! You can make whatever decision you wish about wearing the habit.” St. Pius replied, “Even a pope must obey.”