Within the ancient Kalkulam fortification is the sprawling Padmanabhapuram Palace complex, an early administrative headquarters of the erstwhile Venad rulers. When searching for the roots of the fort and palace at Kalkulam one cannot overlook the Mudaliyar records which hint that the forts in Kalkulam and nearby Udayagiri came into existence just before 1600 C.E, during the reign of Iravi Varma Kulasekharan (r.1592-1609). John Nieuhoff, the seventeenth century Dutch traveller recorded: "Kalkulong (Kalkulam) is a very large city…being on one side strengthened by inaccessible mountains, on the other by wall, the under most of it is of stone, the uppermost of brickwork, in all 24 feet high, the royal palace stands at the west end, being surrounded by a stone wall." Kalkulathu Koikkal the first palace established in Kalkulam by Iravi Varma transformed in the subsequent ages when the later rulers renovated and redesigned the ancient royal abode. In course of time, new blocks added by the successive rulers, reflected varied architectural styles ranging from the traditional to those marked with colonial characteristics, turning the complex into a great architectural marvel.
The palace served as a setting for many significant episodes associated with Travancore history. An important royal proclamation that abolished Mannapedi and Pulappedi was issued from Padmanabhapuram, in 1696. Around 1750, Anizham Tirunal Marthanda Varma (r.1729-1758) rebuilt the palace and fort after which he submitted it to Sree Padmanabha Swamy, the tutelary deity of the royal family. Thereafter the old Kalkulam palace came to be known as Padmanabhapuram. After Marthanda Varma, his successor Karthika Tirunal Rama Varma (r.1758-1798) considered Padmanabhapuram as the principal royal abode. Fr. Paolino, who visited the king in 1783 states: "Padmanabhapuram is a considerable castle not far from Tiruvancoda (Thiruvithamcode), where the king resides and keeps his treasure."Karthika Tirunal's nephew, Avittam Tirunal Balarama Varma, when he assumed charge in 1798, resided mostly in Thiruvananthapuram. In 1809, during the reign of Balarama Varma, Padmanabhapuram and Udayagiri fortifications were captured by the British soldierswho fought against the rebels led by Velu Thampi Dalawa, the Prime Minister of the Travancore State. The subsequent rulers settled in Thiruvananthapuram, where a fortified settlement was established around the Padmanabha Swamy Temple. Even though Padmanabhapuram lost its grandeur with the emergence of Thiruvananthapuram, it cannot be said that the royals overlooked the ancient abode. Until the reign of Swathi Tirunal Rama Varma (b.1813-d.1846), the Navaratri festival was held at Padmanabhapuram and a vicharippukar (manager) looked after the complex.,
The decline of Padmanabhapuram became rapid after the 1850s. The public had no access to the complex and soon the locals came to believe that the palace was haunted! A remarkable description of the palace complex and the popular rumours were reported by artist C. Raja Raja Varma in his 'A Narrative of the Tour in Upper India.' On November 18, 1894, Aswathi Tirunal Marthanda Varma B.A., the First Prince and his party consisting of artist Raja Ravi Varma, and his younger brother C. Raja Raja Varma left Thiruvananthapuram for an expedition to upper India. They reached Padmanabhapuram by evening, where they were received at the palace gates by Dewan Peishkar Thanu Pillai. Later, the party explored the ancient palace complex. "With lantern in hand, we took a ramble in the dusk of the evening, through the dark and deserted halls and the passages of this extensive pile of buildings, startling whole armies of bats, and often mistaking with a shudder our own shadows for ghosts... The simple credulous folk have often heard unearthly noises and seen strange apparitions, and dare not enter the building for any consideration after night-fall, "wrote Varma. After exploring the complex, the visitors retired to sleep,
"dreaming of ghosts and hobgoblins the whole night!"
Thanks to Sree Chithira Tirunal Bala Rama Varma, the last Maharaja of Travancore, Sethu Parvathi Bayi, the Queen Mother, and Dewan Sir. C.P. Ramaswami Aiyer, in 1934, the ancient complex was 'rediscovered' by J.H. Cousins, the Art Advisor to Travancore Government and R. Vasudeva Poduval who headed the Archaeology Department. In 1935, with full-fledged support of the royals, the ancient palace transformed into a museum complex. The reorganisation of the states based on linguistics severed the historic Kanyakumari District from Kerala. However, with timely intervention, Padmanabhapuram Palace and its sprawling grounds came under the direct control of the Kerala State Government. Today, visitors come in thousands to see the palace complex - a vast pile of buildings where memories from a long forgotten age still survive.