By Roman custom, the day added is actually February 24th, with the days following it renumbered. The Romans had marked days during a month: 1st (called calendae--hence "calendar"), 5th or 7th (nonae), 13th or 15th (idus). On these days important events like markets, festivities, and rituals took place. It is possible that in ancient times attempts were made to keep the months in sync with the lunar phases: on occasion an additional day would be inserted inter calendae (hence "intercalary"), i.e. somewhere between those days that should be kept fixed. Now our leap day would repeat the 6th day before the 1st day of March (count including the 1st day itself, as was their custom): hence "bissextile" day, which falls to 24 Feb.
The rule specified by the Gregorian calendar for leap years is as follows:
The logic behind the above rules is as follows:
By adding a day every four years, an average year is adjusted to 365.25 days. However, this still causes a discrepancy with the vernal equinox tropical year. To make the average year more accurate, a leap year is cancelled in each century. This removes 0.01 days to bring the average to 365.24 days. Unfortunately this is still not accurate enough, hence the cancelled leap year returns once every four centuries. That adds back 0.0025 days to bring the average to 365.2425 days.
The adjusted average is still about 0.0001 days ahead of the actual mean interval between vernal equinoxes (365.242375 days). As a result, the Gregorian Calendar will still run about a half-day ahead in 4000 years. Proposals have been made to add an additional rule: e.g. that years divisible by 4000 are not leap years. Such proposals have not been accepted, since the aim of the Gregorian calendar reform was to keep the vernal equinox steady in the calendar and the Gregorian calendar achieves this long-term goal well enough when considering the changing length of the vernal equinox year in the foreseeable future.
Notice that the leap year does not have anything at all to do with leap seconds, which are added occasionally based on actual observations of the rotation of the Earth around its axis.