"[Those who go astray will not harm you when you have been guided]" (5:105). This is not regarding the misguidance of disbelief, but rather regarding the misguidance away from the truth within Islam.

Yahya: On the authority of Abu al-Ashhab, from al-Hasan: that this verse was recited in the presence of Abdullah ibn Mas'ud, and he said: "This is not its [appropriate] time. Proclaim it as long as it is accepted from you, but when it is rejected, then upon you is [the responsibility for] your own souls."

Muhammad said: The meaning is: Allah has only bound you to the concern of your own souls; and when I say, 'Alayka fulanan' (upon you is so-and-so), the meaning is: adhere to so-and-so.

Surat al-Ma'idah from verse (106) to verse (108).