The theory (if you can call it that) arose from a far leftist, who is also a hardcore 9/11 truther. Not a racist, just a did-you-know-government-controls-every-facet-of-our-lives-and-nothing-is-as-it-seems nutjob. Of course, it was also used, initially, by many liberals, mostly a small cohort of Clinton supporters.
This has been said in more ways than one, but simply the idea of something, regardless of any truth, can be just as damning as if it were true. So once a prominent conspiracy theorist floats it out there that Obama is not a citizen by way of a lawsuit, a lot of people latch on to it. It spreads like a fucking wildfire.
I don't think a lot of people that are birthers are racists. They are just dumb enough to think it is a worthwhile cause to attack him on something as so unfounded as a foreign birth. It seems like an easy target, rather than examining his politics, which would require some insight and knowledge. Because Obama did live overseas for a time, and a Kenyan father, just gives them something to point at. Some, however, are racists no doubt.