Why is public transport from/to the airport so expensive?

Actually, the ones you've mentioned - they often have cheaper ways to get there, and in many cities the most common way is very cheap - it's not always expensive.

In your London example, for half the price of the Gatwick Express you can take Southern Rail to get to the airport. EasyJet has an EasyBus from Fulham Broadway which will take you there, and can cost you as little as two pounds!

To get to Heathrow, you can take the tube - a trip which takes an hour, for about 4 pounds! Or even less if you take two buses, and less if you have an Oyster card.

In Vancouver, it costs $3.75 to get to the airport on the Skytrain, from downtown. If you have a simple two-zone pass (like many locals do) it costs nothing extra.

Generally the high price comes down to a few key reasons:

• the cities you mention are expensive cities. The infrastructure, maintenance and so on costs a lot.
• 'tourist tax'. I say this tongue in cheek, but a lot of the costs can be mitigated with local know-how. The Sydney airport train costs so much because of the final charge for entering/exiting the airport - take a local bus to the second train station and you shave a chunk of the cost off.
• distance. Generally (with good city planning) the airports are a long distance from the city. As a result, the maintenance, fuel and time adds up. I refer you again to London - with a single Zone 1-2 tube ticket (a few quid) you can get to the London City Airport. Same city, less distance = less charge.
• profit. If an airport has a captive market, and they can, they could charge.

With regards to the weird taxi/shuttle thing - some people are worried about taking taxis in certain countries and may prefer being with others in a shuttle. Or they're paying for an 'executive' shuttle, for example. Shuttles could also be licensed, while you might get an unmarked taxi.

A good way to mitigate these in advance is to check out the Wikitravel page for the city in advance. Have a look at the 'Get in' section. It'll often describe several ways to get in from the airports, and the cost. For example, from EZE (international airport in Buenos Aires), you could take a taxi (costs!), a shuttle (medium), or the local bus (very little).

Part of the reason might be that often the public transport going to the airports is half-empty, especially that it operates at regular intervals even outside of the normal peak hours. For example the Gatwick Express runs every 15 minutes or so, and if you actually board it, only one third of the seats are occupied. From Gatwick you can also take a normal (Thameslink) train for half the price, which goes from Brighton - but that one will be a lot more packed and you will have to wait a bit longer for that one.

Sydney is an interesting example though (if I am not mistaken here) - you have to pay a lot for a normal commuter train. Even if you board further away than the airport, you pay a fraction of the price. I guess they rip you off because they can.

Travelers are often visitors who have no local knowledge, and are either tired or under time pressure or both, and so the transportation industry takes advantage of them.

Case in point: here in Vancouver, you are fleeced at the airport when boarding the Skytrain. A surcharge is added to fares. But those who have passes or tickets do not have to add the surcharge, only those who are paying cash for their fare at the ticket machine.

Most of those clever enough to have tickets will be returning Vancouver locals or people who work at the airport, rather than visitors.

Some locals say that B. C. stands not for British Columbia, but Bring Cash.

Another reason, beyond the excellent ones already discussed here, is that public transit is typically subsidized by government, often both by direct annual contributions to the transit agency's budget, and by issuing debt for capital expenses that is slowly paid off by some combination of the agency and the government.

For operating expenses (not counting the cost of actually building infrastructure or rolling stock), the extent of the subsidy can be roughly measured with the Farebox Recovery Ratio, which is the percent of operating expenses paid for by fare revenue. As you can see, almost all systems outside the ultra-high-density operators in Asia have rather low farebox recovery rates, often under 50%.

What this means is that any new expansion in service may well cost more to operate than it makes in revenue. This is normally accepted, because that expansion comes with the public good of allowing more residents (taxpayers) to access the system, reduces traffic congestion and pollution, and improves property values (and thus raises property tax revenues) because homes with good transit connections are often more valuable.

But for an airport link, taxpayers tend to balk at subsidizing comparatively well-off travelers headed to/from the airport. There is often a desire for such links to pay their own way in operations expenses, and ideally recoup their construction costs, and so when agencies go to set fares, they often price the airport segment considerably higher than other trips of equivalent distance.

As an example, BART's Oakland Airport Connector in Northern California opened in late 2014. Transit advocates encouraged the agency to set a high fare, so as not to divert funds that are used to serve commuters to the airport. The agency eventually set a fare of $6 (just for the 3.2 mile Connector; the rest of your ride still costs the normal amount). Even at this high fare, there's considerable question about the Connector's ability to pay for itself, let alone pay back the hundreds of millions of dollars it cost to build.

TL;DR: Transit is usually a lot more expensive than you think, but is often heavily subsidized. When the subsidies are reduced or eliminated, as often happens with airport links, the fares look far more expensive by comparison.

I can't say for Australia, but in Western Europe the public transportation is very expensive.

As opposite, in East Europe and the former USSR public transportation is usually much cheaper.

In Warsaw there's a city train connection with the airport. Currently a single ticket (which can be used to travel to the city center) costs 4.40 PLN (about 1 Euro). Taxi would cost you at least 5 times more.

As comparison in Nuremberg public transport for city range (including Metro to airport) costs 2.50€. This is the same as normal public transport. I don't know the exact prices in Berlin now, but there also public transport to the airport costs exactly the same as to other locations (it is, however, zone 3, so it would cost slightly more than transport inside city center).

Generally your question could be a bit modified and the answer would be: Because you were in places where public transport is so expensive.

There are two basic, interrelated reasons for this.

The first is that most people don't want airports near their "back yards" because of the noise, pollution and small (but real) fear of crash. As a result, airports are located in "remote" parts of a city, typically 20 miles or so from the center of town. Just this fact alone makes for a long, costly trip to the airport.

The related issue is because of the airport's remoteness, there are few alternate means of transportation to and from it. It's not like the train station (typically in the center of town), where numerous buses (and sometimes subway trains) go to. The (relatively) few transports that go to the airport can "charge what the market will bear."

With SFO and BART, there is a significant surcharge for using the airport station; it is cheaper to travel further down the line. (From the station I use, the fare is now about $9.00; from downtown San Francisco it is less.)

Some reasons: the station had to be retrofit into the airport. And the airport wants the money, because it turns out they make more money from parking fees than airplane landing fees.

There is also a surcharge for the new connector from BART's Coliseum Station to OAK. This also was a retrofit, long after both the transit system and airport were built.