Qasur is plural of Qasr (قصر) which means fortress or castle in Arabic. The place was named so because of the twelve forts built by Pashtuns settlers in 16th century, majority of whom belonged to the Khweshgi (also spelled as Kheshgi) tribe of Pashtuns. The twelve forts were named after the heads of the twelve clans of Kheshgis. Seven of the twelve forts were still standing in 1871 as reported by Alexander Cunningham.
Those Pashtun settlers had accompanied the army of Mughal king Babur in 1526 and had fought on his behalf against the army of Sultan Ibrahim Lodi. As a reward for their services, Babur granted them the present territory of Kasur as jagir. It is said that Sher Shah Sur did not hold the Khweshgi Pashtuns of Qasur in favour, on account of them siding with the Mughals at Panipat. On the other hand, they were regarded with favor by Mughal emperors and many of them attained high ranks in the Mughal nobility.
The city of Qasur built and ruled by Pashtuns, was massive. William Barr saw the extensive ruins of Qasur (destroyed by Ranjit Singh) in 1839 and writes: “Kusoor, a large and ancient town, that in former days must have covered an extensive area, as its ruins are interminable”.