linthead
n.— «Almost always it features early on the fact that his mother’s family worked in Revolution Mill in Greensboro, that because of his grandmother’s asthma his family moved into the McAdoo Heights neighborhood and his mother switched from the mill school to the city school system. There, he recounts, she was teased as a “linthead,” a derogatory term for mill workers.» —“Neal takes campaign one voter at a time” by Mark Binker News & Record (Greensboro, North Carolina) Mar. 30, 2008. (source: Double-Tongued Dictionary)
I grew up in nearby Burlington, NC and mill workers were called “lint dodgers”. Depending on who said it to whom, it was either an inside joke type of thing or a derogarty thing.
I grew up and still live in Kannapolis, NC, a former mill town. I heard the term “linthead” from my father and other former mill workers, and it was mostly considered a derogatory term – or, as Jeff mentions above, an inside sort of joke.