Can you still travel to America on the visa waiver program if you have been to Iran in transit?

Some travelers are not eligible for ESTA. The US Customs and Border Protection FAQ says those travelers include:

Nationals of VWP countries who have traveled to or been present in Iraq, Syria, Iran, Sudan, Libya, Somalia, or Yemen at any time on or after March 1, 2011 (with limited exceptions); and

where those exceptions are

to represent your program country on official military orders or official government business

It depends on what the US definition of "travelled to or been present in" is, but taking that to be literal "set foot in the country" (the safest definition from your perspective), you and your wife would no longer qualify for an ESTA and would need to do a full visa application.

With the disclaimer that I'm not sure how the CBP interprets the rules for this edge case, for most practical purposes, if you never went through immigration, you've never been to Iran.

So in your shoes, I would say "no" in the ESTA application, and in the unlikely event of being questioned on arrival, I would simply state the above.

The other option would be to state "yes", which will likely cause the ESTA to be rejected, and then apply for a visa and explain the situation. However, while you will very likely get the visa this way, it may take so long that you miss the trip -- and if you've failed ESTA once, you now need to apply for a visa for the US for the rest of your life.

I must agree with jpatokal with a small correction: unless you went or should have gone through immigration, you haven't been in Iran (otherwise illegal border crossings wouldn't count). Otherwise really strange questions arise. Like, if you have sailed through the Strait of Hormuz it is entirely possibly you have crossed waters Iran claims theirs but the United States disputes these claims. So, according to the United States have you been to Iran if you have been only on territory the United States doesn't recognize as a territory of Iran...? If your plane does an emergency landing in Iran, do you lose your ESTA rights for life? If your plane does a refuelling stop where you can't get off and noone can get on, have you been to Iran? The possibilities are endless...

Even more importantly, what happens if you enter a consulate of Iran? What definition can you find aside from an actual border crossing that differentiates from the territory of the embassy and the country itself?

So my wife got a response back from ESTA after applying that the authorisation is still pending even though it's been more than 72 hours since she applied. This was the response from the CBP officer:-

Your application is pending additional administrative processing due to your previous travel to Iran. Applications that show travel to a restricted country can take considerably longer than the normal 72 hours to review. Alternatively, we strongly recommend applying for a visa at the nearest U.S. Embassy or Consulate.

The trip is in just under a week's time so it looks like she could miss out unfortunately