What are "Signal Failures" on The Tube and why do they happen so often?

In lieu of a link-only answer...

A signal failure can refer to a number of situations... It can be down to a track circuit failure which means that the signaller cannot guarantee the safety of the train as he is unable to see on their screen or panel that the section of line is clear of any other trains. It could be down to one of the Train Protection Warning System failing which can cause trains to stop when it is not required, these are in place to stop drivers passing signals at danger and risking collisions, it is also in place on certain bends of tracks where there is a speed limit, this is to stop tragic accidents like the one recently in Spain. it can be the set of points which switch you from one line to another have failed in their position, there are thousands of different failures, it can be level crossings have failed.

Much as traffic lights regulate and control the flow of traffic on the roads, railway signals perform an analogous function on the railway network - not to permit crossing (which is illegal), but to prevent railway trains running into each other's rear. As a safety measure, the default position of any railway signal is "danger", that is, indicating a hazard ahead. Any signal giving an ambiguous display (one where the meaning is unclear) is, in the instructions of train drivers, to be regarded as "danger" until proved otherwise. In the days when signals were given using a cable-operated baton, once the baton froze solid in the "all-clear" position, causing a terrible train crash. Signals were afterwards directed always to stand in the "danger" position when at rest, a system known as a fail-safe

Source http://www.theguardian.com/notesandqueries/query/0,,-201845,00.html

For the other parts of your question, they appear to happen often because the system is large, complex, and old. Plus we don't want to pay for it. Overall, the record is impressive. Last year for example the DLR achieved the highest punctuality rate in 26 years.

For your last question, they will never be ENTIRELY eliminated, but year-on-year reductions are averaging around 10%.

Source https://www.tfl.gov.uk/cdn/static/cms/documents/annual-report-2013-14.pdf

Finally, if you want to make representations, you can contact their Stakeholder Engagement team and get the name of the passenger representative that handles stakeholder issues for your routes StakeholderEngagement@tfl.gov.uk.

A signal failure is the failure of a signal and associated equipment.

The most important thing to realize is that a train at full speed cannot stop on sight so operators need to know that the way ahead is free. There are several different systems to deal with this difficulty and regulate traffic on railroads but most of them depend on signals in one way or another. If a signal is not in the “all-clear” position, the train has to stop or at least slow down.

Presumably the signal itself can fail for all the usual reasons (power outage, broken lamp, vandalism…) and it is also typically designed to stop traffic if for some reason (malfunction in a sensor used to detect the passage of trains, communication problem…) it cannot be determined whether the next block or section of railroad is free or not.

But unlike traffic lights on the road (which are less numerous – relatively speaking – and can easily be ignored by drivers or overruled by the police), you can't just ignore a railway signal until the problem is solved. There are big safety implications and railway companies tend to be very hierarchical so procedures have to be followed and an individual operator certainly cannot let a train run on his or her own initiative. Crossing a “closed” signal can mean dismissal.

When traffic is very dense, everything has to work perfectly and any issue will have long-lasting effects down the line thus increasing the impression of unreliability. Beyond that, it is far from trivial to run a train network but ultimately regular problems often boil down to a lack of investment in the maintenance and modernization of the network.

I don't know much about the London underground specifically but Wikipedia suggests part of it relies on a signalling system installed in the 1940s and it is being upgraded, which suggests that signalling has been recognized as a problem.

I'll try to explain in layman terms. Also see Wikipedia article for more details.

The idea is that the whole track is divided into sections (several hundred feet long in underground). When a train enters any section its wheels electrically connect the rails and this can be electrically detected and semaphores can be adjusted automatically and the data on track being occupied can be seen by nearby trains.

If there's a train on some section then the section is considered occupied and the semaphore in front of that section will display red (prohibitive) light - autopilot-like system in the train will detect that electrically and stop the train before that light. One or several sections before the occupied one will have semaphores displaying yellow (warning) light - autopilot-like system in the train will detect that electrically and slow the train down in those sections. Sections "far enough" from the occupied ones will have semaphores with permissive light and there the train can roll at high speed.

Now note that there's a lot of trains on the same track and so "far enough" can be just two or three sections between trains. Should any of those trains suddenly stop all the rest will have to queue. Should any piece of wiring used for detecting track being occupied fail all the other components immediately act paranoid and assume the track is occupied and stall the trains. One that happens the pilot has to contact the dispatcher and manually run the train being verrrry careful and paranoid because the tunnels have a lot of curves and visibility range is very poor.

So basically a minor fault in that complex system immediately makes everything act paranoid and that slows thing down greatly.