After having turned away from God by sin, penance is essentially acts by which we turn back to God and bring forth fruits worthy of repentance. St. John the Baptist and the Apostles preached the need for such acts (for example, see Matt. 3:8, Luke 3:8, and Acts 26:20).
Prayer is the most basic and obvious fruitful act of turning to God, which is why it has become the most common form of penance. Other acts, like those of self-denial, show our willingness to turn away from the selfish acts of sin. Acts of charity, like alms giving, help us to turn to God through love of neighbor, especially as our selfish sins often to harm to our neighbors.
They are all good acts of penance, but prayer really is the most simple and fundamental.