"They said, 'Should we believe in you, while the lowest of the people follow you?'" meaning: the base and lowly ones.