The fort was built by the Portuguese in 1618 under Philip De Olivera following the Portuguese invasion of Jaffna. The fort is located near Karaiyur. Due to numerous miracles attributed to the statue of Virgin Mary in the church inside the fort, Jaffna Fort was named as Fortress of Our Lady of Miracles of Jafanapatão (Fortaleza de Nossa Senhora dos Milagres de Jafanapatão). It was captured by the Dutch under Rijcklof van Goens in 1658 who expanded it. In 1795, it was taken over by the British, and remained under the control of a British garrison till 1948. As the only large military fort in the country, due to the presence of only government and military buildings within its ramparts, it was garrisoned by a detachment of the Ceylon Army.
With the onset of the Sri Lankan Civil War it came under siege on several occasions and was the scene of pitched battles. From 1986 to 1995 it was under the control of the LTTE during this time the LTTE destroyed several of key features to stop the Army getting control due to the site being used to stage attacks but it was recaptured by the Sri Lanka Army in 1995 after a 50 day siege during Operation Riviresa. It was also vandalised by locals to rebuild houses damaged from the war .Today it remains garrisoned by a detachment of the Sri Lanka Army with limited access to visitors and is being renovated with Dutch funding.