First off, it isn't racial, it's cultural.
Second, it's unclear is the origins of the term are self identified or imposed from the outside. It first appeared in print in The New Yorker, but the author states it was the term the locals used to describe themselves.
Finally, while it has some derogatory aspects [poor, illiterate] it's also embued with positive attributes [self sufficient, free, hardy].
Bill Parcells to Phil Simms: Son, it breaks my heart that my expectations for you exceed those you have for yourself.