Carter worked on his parents' Georgia peanut farm as a boy. By the time he was 10 years old, Carter was hauling produce into town to sell.

He left Georgia to attend the U.S. Naval Academy. After serving in the Navy for several years, Carter moved his family back to the farm when his father died. A terrible drought devastated the farm in 1954, leaving him a profit of just $187 ($1,654 in 2014 dollars) that year.

Carter successfully turned the farm around before entering politics.