He hailed from Duling 杜陵 (near modern Xi'an 西安, Shaanxi) and was an excellent scholar in youth. His father Han Yi 韓義 had been executed because of his remonstrance against Prince Ci of Yan 燕刺王 whom Han Yi thought to plan a rebellion. Emperor Zhao 漢昭帝 (r. 87-74 BCE) remembered the loyalty of Han Yi and therefore appointed Han Yanshou Grand master of remonstrance (jian dafu 諫大夫) upon the recommendation of the powerful general Huo Guang 霍光.
He later was made governor (taishou 太守) of the commandery of Huaiyang 淮陽, then of Yingchuan 穎川 and finally of Dongjun 東郡. With his Confucian education he adapted the political thought of Confucius into his style of administration. The emperor therefore appointed him governor of Zuofengyu 左馮翋.