You are trying to apply literal and metaphorical to fit a theology. Yes, Peter was mobile, His office that he was chose and appointed too was not. The Apostles were the foundation. Christ told about the wiseman that built his house on rock, and of the man that built his house on sand. Which foundation would Christ choose to build His house?
The answer to that is exactly what you need to consider.
Matthew 7
24"Therefore everyone who hears these words of mine and puts them into practice is like a wise man who built his house on the rock. 25The rain came down, the streams rose, and the winds blew and beat against that house; yet it did not fall, because it had its foundation on the rock.
Matthew 21
42Jesus said to them, “Have you never read in the Scriptures:
" ‘The stone the builders rejected
has become the capstone;
the Lord has done this,
and it is marvelous in our eyes’?
43"Therefore I tell you that the kingdom of God will be taken away from you and given to a people who will produce its fruit. 44He who falls on this stone will be broken to pieces, but he on whom it falls will be crushed.”
45When the chief priests and the Pharisees heard Jesus’ parables, they knew he was talking about them. 46They looked for a way to arrest him, but they were afraid of the crowd because the people held that he was a prophet.
1 Corinthians 3
11For no one can lay any foundation other than the one already laid, which is Jesus Christ.
1 Corinthians 10
4and drank the same spiritual drink; for they drank from the spiritual rock that accompanied them, and that rock was Christ.
Romans 9
33As it is written:
“See, I lay in Zion a stone that causes men to stumble
and a rock that makes them fall,
and the one who trusts in him will never be put to shame.”
1 Peter 2
4As you come to him, the living Stone—rejected by men but chosen by God and precious to him— 5you also, like living stones, are being built into a spiritual house to be a holy priesthood, offering spiritual sacrifices acceptable to God through Jesus Christ. 6For in Scripture it says:
“See, I lay a stone in Zion,
a chosen and precious cornerstone,
and the one who trusts in him
will never be put to shame.” 7Now to you who believe, this stone is precious. But to those who do not believe,
“The stone the builders rejected
has become the capstone,” 8and,
“A stone that causes men to stumble
and a rock that makes them fall.” They stumble because they disobey the message—which is also what they were destined for.
Deuteronomy 32
4 He is the Rock, his works are perfect,
and all his ways are just.
A faithful God who does no wrong,
upright and just is he.
…
18 You deserted the Rock, who fathered you;
you forgot the God who gave you birth.
…
31 For their rock is not like our Rock,
as even our enemies concede.
Psalm 18
31 For who is God besides the LORD ?
And who is the Rock except our God?
Psalm 31
2 Turn your ear to me,
come quickly to my rescue;
be my rock of refuge,
a strong fortress to save me.
3 Since you are my rock and my fortress,
for the sake of your name lead and guide me.
Psalm 62
6 He alone is my rock and my salvation;
he is my fortress, I will not be shaken.
7 My salvation and my honor depend on God;
he is my mighty rock, my refuge.
8 Trust in him at all times, O people;
pour out your hearts to him,
for God is our refuge.
Psalm 78
35 They remembered that God was their Rock,
that God Most High was their Redeemer
Psalm 92
15 proclaiming, “The LORD is upright;
he is my Rock, and there is no wickedness in him.”
Psalm 94
22 But the LORD has become my fortress,
and my God the rock in whom I take refuge.
Isaiah 28
16 So this is what the Sovereign LORD says:
"See, I lay a stone in Zion,
a tested stone,
a precious cornerstone for a sure foundation;
the one who trusts will never be dismayed.
2 Samuel 22
2 He said:
"The LORD is my rock, my fortress and my deliverer;
Look what the Apostle taught the TRUTH that is the foundation:
Ephesians 2
19Consequently, you are no longer foreigners and aliens, but fellow citizens with God’s people and members of God’s household, 20built on the foundation of the apostles and prophets, with Christ Jesus himself as the chief cornerstone. 21In him the whole building is joined together and rises to become a holy temple in the Lord. 22And in him you too are being built together to become a dwelling in which God lives by his Spirit.