Idumban bore the hill slung across his shoulders in the form of a Kavadi, one on either side using the Danda. Of Brahma as the rod and the snakes of the earth the ropes to tie thern, and turned towards Podikai Hill, the mountain abode of Sage Agastya. When he came to Palani he was bewildered to the route, when he received guidance from a king passing that way mounted on a horse. He went in the direction shown to him, but being fatigued, placed the Kavadi at Palani and took rest.

When Idumban was resting at Palani, Lord Subrahmanya was coming away from his parents, discomfited in a contest with his brother Ganapati. The legend says that Iswara, seated with Parvati, once called to Him His two sons Ganapati and Subrahmanya, and addressing them, said: "I -have a rare fruit in my possession and shall give it to that one of you, who goes round the world and returns to me first." Subrahmanya, eager to win the prize, started on his peacock at once, which flew with its Divine Burden quicker than lightning, while Ganapati sat quiet until his brother disappeared out of sight and then, slowly rising, went round his parents and asked for the fruit, saying: "All the worlds that are, that were and that will be are within you and, by going round you, I have gone round all of them. Therefore the fruit is mine." Parvati and Parameswara were delighted with the reply and gave him the precious fruit (pomegranate). Long after this was over, Subrahmanya came sweating, on his peacock, only to find that he had been outwitted.

The story should not be taken literally. It is the philosophy in it, which is essential. The lesson briefly stated is, that God being known, everything else is known and no study of the external world, however comprehensive that may be, can ever yield us the precious fruit of wisdom. "Knowledge of course will come, but wisdom with its peace and bliss will linger on the shore." Ganapati outwitted his brother by sheer wisdom. Lord Subrahmanya was not very much satisfied with this explanation of his father and in anger came away to Tiruvannagudi, the place at the foot of the Sivagiri Hill. Then he came to Sivagiri Hill itself. When Balasubrahmanya got angry for not receiving the fruit and went into seclusion, Siva pacified him saying that the fruit was not separate from him (Subrahmanya) and he himself was the fruit [Pazham (Palam) -fruit; nee-you]. Lord Subrahmanya forsook all worldly garments and with kaupeenam. (Just a shred of cloth to cover his nakedness) and a baton staff in hand, he became a recluse. Finding the two hills nearby, he got on the top of one and rested. When Idumban wanted to lift his Kavadi to continue his journey, he could not lift it, as Subrahmanya had already got on the top of one and it was heavy. He investigated the reason and, finding an intruder on the top of the bigger hill, he challenged him to get down.

Idumban prayed to Muruga that he might be allowed to stand ever at His portals as dwarapalas, and also that whoever offered vows with the Kavadi be blessed. Lord Muruga granted these boons to him. Since then pilgrims to Palani bring they’re offering on their shoulders in a Kavadi. The hill is 450 feet in height, reached by a flight of 659 steps. The idol is composed of Navapashanam. (Nine different kinds of poisons) blended together in certain rare proportions, the resultant substance resembling wax. The mixing of the nine poisons was done by a Muni called Bhoga. Some say that the constituent of the image is an amalgam of nine minerals.The body of the deity is not smooth and seems to have been either scraped out or corroded by age. It is said that some Siddha Vaidyas of the locality, who knew of the miraculous curative and preservative properties, of the image, scraped the body of the deity in the unknown Apparently unnoticed and used the same for curing otherwise incurable diseases of men. The touch of the body is believed to work cur, wonders.