This was when the polytheists were banished from the Sacred Mosque.

Muhammad said: "bearing witness" is a circumstantial qualifier (hal). The meaning is: It was not for them to maintain the mosque while they were acknowledging their disbelief.