James K. Vardaman was my great-grandfather (he was also later a US senator for MS). My brother has done a lot of family research, and though "Vardhaman" is I think Hindi/India, it seems my family on my father's side (and further back) is somewhere in the Scandinavian area. I heard many years ago it's possibly a misspelled variation on "Waterman" - which suggests some kind of boater or fisher. My brother scanned a copy (from the MS Dept Of Archives) of a payslip to an ancestor Jeremiah Vardaman for his service in the War Of 1812. So parts of my family have lived in MS before MS was even a state.
It's a pretty uncommon name, but the Vardamans in MS are mostly related to me, some also around Birmingham, AL. There are also Vardamans in Indiana, but not sure if they are related to me.
I'm not sure if it's a Southern thing, but I'm actually related to at least one distant cousin who has the first name of "Vardaman". Seems like back then people would swap first and last names.