Why do airlines now charge for flying standby in the US?

The "Why" is simple: airlines who stopped offering free standby and instead offered guaranteed seats on a different flight for a change fee saw an increase in revenue from doing that, and so continue to do it.

As for why it increases their revenue, there are four upsides:

• first, the fee itself. $100, $200 - in a time of $99 cheapest fares, the fee itself is good money for them
• second, passengers change less or change further in advance. Either way the airlines can plan a little better - how much time to allow to fill the plane, even what size plane to use or which flight of the day to cancel if they have one less plane than usual
• third, the airline gains a "perk" (free changes at the airport) to offer frequent flyers or to differentiate on one high-competition route. I have seen this in action
• fourth, and most important, passengers now tend to buy the flight they actually want, even if it costs more, rather than buying a cheap one and planning all along to switch to an earlier one

Savvy business travelers in the 80s always booked the last flight of the night and then tried to sweet talk their way onto an earlier flight, depending on when they got to the airport. That way, they were sure they wouldn't miss their plane, you see, but could still get home early if the day finished early. As prices for the cheapest and most expensive seats diverged, this became more about money than about being sure you wouldn't miss your flight. The 6:30 pm flight everyone wants is $500 and the midnight flight is $200. So buy the midnight, and see if you can get on the 6:30. This behaviour assuredly costs the airline money - the $300 extra they could have charged you for the popular flight if there were no fees. Saying it costs them nothing once you're at the airport and the seat is empty ignores the fact that you're going to fly again and you're going to make a decision about what flight to buy, based in part on your experience right now.

People were in the habit of playing these games. Airlines knew it. And it was messing with their revenues and their plans. So in come the change fees. Now some people still make changes, but generally it's because of a real change in their life, and not because they were playing chicken to see if they could get home both as cheaply as possible and as conveniently as possible. The result for the airline is they sell more expensive fares to those who want to be sure they'll fly at a particular time.

Those are the ups. The down is that you are irritated at the airport for a while. Will it make you fly another airline that this one didn't give you something free? I doubt it. For one thing, you can't go switching to another airline that does offer this, because none of them do. For another, frequent flyers tend to choose airlines based on the direct flights they offer, the "hard product" (seats etc), the service, lounges, and so on. They aren't going to switch over change fees - if anything they might strive more for a high-tier status in order to achieve free changes, or buy more expensive tickets that offer free changes. Frequent flyers love to grumble about their airline but they rarely actually switch. And infrequent flyers by definition are low volume - whatever decisions they make are unlikely to have much impact on the airline.

So there you go: lots of ups, little or no downs. The "obvious" logic in the moment of giving you the seat is far far outbalanced by the long term logic of encouraging you to not bother buying it and hope each time to be given it. (See also free upgrades since business class is empty anyway - a strategy which leads inexorably to nobody ever buying business class tickets.) Running an airline is insanely difficult - that's why they're constantly failing - and the logic of what to charge for is never as simple as it appears to be at first glance, thanks to a huge pool of customers who will change the behaviour in response to your policies, trying to lower your revenue (their costs) as much as they can.

They are providing a service and the current airline revenue model is to charge for services. There is more to it than just putting you on the plane.

The gate agent has to modify your reservation to reflect the new flight(s) which would normally incur a fee if you change it by phone (except a few higher fare buckets). The pilots need to recalculate their load and balances for take off, fuel, etc. The catering needs change, though this applies more to the later flight, since the standby flight is likely already catered.

While the later two often have a certain tolerance and one passenger might not require a change, they are still time consuming aspects for which airlines want compensation. And the load and balances calculations are a very important safety aspect for your flight.