That is because they said to those who disliked what Allah sent down: "We will obey you in some of the matters." Meaning: in associating partners with Allah, they agreed with them upon shirk (polytheism) in secret, "and Allah knows their secrets."
Muhammad said: Whoever recites it with the fatha (vowel sign) on the alif, it is the plural of 'sirr' (secret).