Born in Portsmouth, New Hampshire, Thomas Bailey Aldrich moved with his father to New Orleans, Louisiana at the age of three. He remained there until age thirteen, when his father's impending death prompted Aldrich's return to New Hampshire and his mother's household (Parker). At age sixteen, Aldrich started working as a clerk for his uncle, Charles Frost: "While working over the books of the firm, his mind was often busy with themes outside of the commission house, all leading towards a literary career" (Hemstreet 218).
Charles Kingsley was one of several men who were associated with the Fred Gray Association, "a loose confederation of young men who seemed anxious to explore new possibilities of male-male affection" (Folsom & Price “Walt Whitman”). E. Miller explains that “during this period Walt [Whitman] was intimate with a group he called the Fred Gray Association. Not much is known about this circle. Gray, the son of a noted New York physician, took a medical degree after the war. Nathaniel Bloom became a successful merchant. Hugo Fritsch was the son of the Austrian consul.