Does all the subsequent itinerary on a round-the-world ticket get invalidated if you are denied an entry?

In this case, if you are denied an entry for a country, will you get deported to the departure country

If you are denied entry, you are not being deported. Only a person inside the country can be deported. You are being refused entry.

In this case, if you are denied an entry for a country, will you get deported to the departure country or to the next (or another) country

If you are refused entry, the immigration authorities will advise the inbound carrier where you should be sent. They may be happy to send you on to the next country on your itinerary. In principle, they could insist you be sent back to the country who issued your travel document. I can imagine that some countries wouldn't really care as long as you go away; other countries may be more fastidious.

In this case, if you are denied an entry for a country, will you get deported to the departure country or to the next (or another) country by changing the date of the next flight at the border control?

Per the IATA Ticketing Handbook, the final inbound carrier is responsible for re-ticketing an inadmissible passenger to the destination advised by the immigration authorities. If you have a ticket for an onward flight out of the country, then in principle (by the IATA rules on denied entry), the inbound airline should take ownership of the ticket and re-date it for immediate departure. Any change penalties or restrictions on use will be ignored.

And is the subsequent itinerary still valid or does the one denial make all of the subsequent flights invalid?

The remainder of the ticket should not be affected by a date change.

However these rules were drawn up in the days of paper tickets when it was relatively straightforward to take the paper ticket and change the flight dates with a revalidation sticker. The ticketing carrier would never find out. You would then take your ticket to the check in desk for the next airline, which would be obliged to accept it as a valid transportation document.

These days with electronic tickets it would probably require some interaction with the ticketing carrier — ANA in your case — and the onward carrier. The inbound carrier, whose contracted staff at a distant outstation may simply wish to go home or begin their work for their next airline, may or may not be willing to entertain this kind of complexity, and may know less about the correct procedures for inadmissible passengers than Travel Stack Exchange does. ANA has a good reputation for dealing with complex changes but you may find that their ticketing experts are asleep when you need them most.

Meanwhile, as an inadmissible passenger stuck in some transit lounge somewhere sans passport you may not have the moral authority/gravitas to insist that the ticket be changed to your specification.

My advice, if you are at risk of being an inadmissible passenger, is to obtain all your visas in advance of departure. Most RTW tickets provide for a full refund in the case an embassy refuses to issue a visa.