The ancient texts describe Rudra Mahalaya as a magnificent Shiva Temple with a three-storeyed 'shikhara', 1600 pillars, 12 entrance doors, central 'mandapa' and porches on east, north and south and sanctum in west. Around the temple, there were 11 shrines of Rudra.
The eastern gate was adorned with beautifully carved 'Toran', with a flight of steps leading to the Saraswati river. Ulugh Khan and Ahmed Shah of Ahmedabad ruthlessly destroyed this magnificent Shiva Temple, in the late 13th century or early 14th century. The cells on the surrounding walls were converted in to a mosque.