How to transit through Delhi without a visa?

As is the case with many countries, India also follows the Transit Without Visa (TWOV) rule for layovers not exceeding 24 hours as you will be routed through the airport's transit area without having to clear immigration. However, this is valid only for tickets made under a single booking. I say 'single booking' instead of 'airline' because there are many airlines that codeshare such as United Airlines and Virgin Atlantic (for travellers inbound from US/Europe) where United / Virgin fly from US / Europe and then the second leg (most popular ones being Nepal, Sri Lanka, Thailand, Singapore, Hong Kong) is operated by Jet Airways under codeshare. In that scenario, you don't need a transit visa as the single booking allows you bags to be transferred.

For every other scenario, yes, you officially do need a transit visa. This is even though all airlines fly out from the same terminal, in Delhi's case, Terminal 3. (This used to be different earlier as Jet Airways, Air India, and other airlines operated out of an 'International Terminal' and the rest from a 'Domestic Terminal'.) As is standard in many airports, if your booking are with different airlines, then you'll need to collect and check-in your luggage yourself.

Is it possible to have airline staff bypass that loop for you? Yes and no. No, there's nothing which officially says they'll do it or any defined procedure that you can ask an airline for it advance. But then, Jet Airways is one of those airlines which bends over backwards to please anyone with Platinum or Gold level on their frequent flyer programme (or equivalent levels on partner airlines) and it's not uncommon to see Jet Airways staff chaperoning their 'priority' passengers through security checks or checking in baggage on their behalf. If you're travelling economy class, don't expect it. (Although, again, Jet Airways is one of those airlines which will chaperone you through security checks if you have if you are running late for a connecting flight.)

As for other airlines, don't expect similar service from Air India - even though it's the Indian flagship carrier. It's the only Indian airline with unionised staff (it's a government-owned airline) and service levels are typically not as good as other airlines.

SpiceJet and IndiGo are both low-cost, point-to-point airlines with no frequent flyer programme, no way of knowing whether you're a priority passenger, and is often with LCC / point-to-point carriers their workflow is geared more towards people checking in their own bags for different flight legs anyway. I wouldn't expect them to be able to liaise and transfer bags from another airline for that reason.

TL;DR: While it is possible you may have heard stories of people being helped by airline staff to have bags transferred to connecting flights, YMMV, don't count on it. Get the transit visa in advance. Because if it turns out airline staff can't help you (short connection time? cannot liaise with another airline's baggage staff? other reasons?), you'll really be in deep trouble because India doesn't issue visas on arrival (except for a few countries).

I went through this in Oct 2012. After reading a few threads on line we decided to get an Indian transit visa "just in case". This turned out to be a waste of time and money. Transferred from a BA flight into Delhi to an IndiGo (discount airline) flight to Kathmandu. We are Canadian citizens. Upon arrival in Delhi there is a transfer desk that arranges the baggage transfer between airlines. We tried to go through customs with our transit visa to collect the bags ourselves and re check them for the next flight and were sent back up to the transfer desk. Worked the same way on the route home. One note, if you have a long layover in Delhi, you may be sitting in the small transfer lobby with no amenities for a long time!

I went through today (March 2013) and was told the opposite at the transfer desk. I was asked if I had a visa at the transfer desk and when I had I was told to go through immigration and check in landside. This is why I had a visa.

Single booking/itinerary is not what will get your bags checked through. Its whether the airlines concerned have an interline agreement for luggage. If not some may check your bags through anyway but don't count on it.