He hailed from Beidi 北地 (modern Heshui 合水, Gansu) and was a son of the Marquis of Pingqu 平曲侯. As a cavalry officer he became a secretary (sheren 舍人) of Prince Liu Che and was at the same time the brother-in-law of the prince (who was married to Lady Wei 衛氏).
When the prince mounted the throne (known as Emperor Wu Emperor Wu 漢武帝, r. 141-87 BCE), he was appointed Chamberlain for the Imperial Stud (taipu 太僕). Later on he was promoted to General of the light chariots (qingche jiangjun 輕車將軍).
In 124 he became General of chariots and cavalry (cheji jiangjun 車騎將軍) and followed general Wei Qing 衛青 in his expedition against the steppe federation of the Xiongnu 匈奴. Gongsun He captured a Xiongnu khan and was rewarded with the title of Marquis of Nanjiao 南窌侯. He was later stripped off this title and was sent out again campaigning against the Xiongnu, but without success.
In 103 BCE he replaced Shi Qing 石慶 as Counsellor-in-chief (chengxiang 丞相) and was bestowed the title of Marquis of Geyi 葛繹侯. His son Gongsun Jingsheng 公孫敬聲, appointed Chamberlain for the Imperial Stud, behaved inappropriate and was put into jail. Gongsun He thereupon arrested a certain Zhu Anshi 朱安世, a knight-errant who was to die for Jingsheng. Yet Zhu Anshi submitted a memorial to the throne that accused Jingsheng of an illicit relationship with Princess Yangshi 陽石公主, and of sorcery against the emperor. Both father and son were thereupon sentenced to death and were executed.