If a country does not require a visa on arrival, how do they keep track of the traveller?

The visa on arrival / visa free entry does not mean that you bypass immigration. It just means that you don't need advance permission (what a visa actually is) to enter the country.

It has no relation with stamps or recording the entry of the visa-free person. Each country has their own mechanism of tracking visitors; and if you are visa exempt you are not exempt from such tracking.

The US allows visa-free entry for some nationalities, yet everyone has to go through the immigration officers before entry into the country. In the case of the US, you are given a stamp that details how long you are permitted to stay for that period.

The Schengen zone has similar procedures, where you are given a stamp on arrival and then another on departure (the US has no departure formalities).

Yet another example is Dubai. They have visa-free entry for many countries, and depending on where you are coming from you can either use the biometric lanes; or use the normal immigration lanes. Each person is given a stamp to track their entry and exit.

Of course, if you are citizen of a country you are also entitled to visa-free entry into your country of citizenship. Even these entries and exits are stamped.

Tracking of travelers is very important - it allows jurisdictions to forecast and track tax and other income from travelers, it allows them to do capacity planning for public works and infrastructure, and in some areas it allows them to claim federal benefits.

In short, tracking of visitors has very little to do with visas.

Now, each country keeps a record of who entered and who is overstaying; usually done by recording the person's entry (and/or exit) in a central computer system which is accessible to law enforcement and immigration.

The system isn't foolproof though. People who are entitled for visa-free entry do sometimes overstay. If they are caught (either in-country by law enforcement or on exit from country) they are subject to the immigration laws of the country (this can be anything from a slap-on-the-wrist fine and a promise not to do it again, to a hefty fine and jailing/deportation all the way up to a ban). In almost all cases, the person then loses the visa-free privilege and sometimes they are barred from entry.

Countries (or groups of countries with shared open borders, such as Schengen) have different entry requirements depending on their laws, your nationality, the purpose and duration of your visit, and other particulars. Meeting these requirements may require getting a visa in advance, receiving an e-visa online before traveling (or something like an ESTA or eTA which isn't called an e-visa for diplomatic reasons, but basically is one), receiving a visa-on-arrival at the airport, and entry without a visa.

It's important to carefully research the requirements in advance, because there are usually a lot of detailed rules about when they apply. There may be exemptions for people of certain nationalities, special rules for people who want to transit the country (with or without leaving the secure international zone of the airport), different procedures for long visits or those coming to the country to study, to work, etc... Typically, airlines will ensure you meet these requirements before you are allowed to board the plane, but the final decision lies with the authorities of the country concerned.

Whatever requirements apply, when you arrive in virtually every country (except when traveling between countries in one of a handful of areas with open borders), you'll encounter a border checkpoint. An immigration officer will take your passport, maybe ask a few questions, and decide whether to let you in. If you are admitted, he/she will typically stamp your passport with an indication of how long you are allowed to stay, and in most countries, will record your details in their computer system. If you stay beyond the allowed time, or do something like work that you are not permitted to do, you are subject to that country's laws and there may be legal consequences.

In short, it is possible for someone to try to "walk into a country and disappear" whether they have a visa or not. This has nothing to do with whether you needed a visa to enter the country; either way, you have the same opportunity to try to disappear and they have the same opportunity to catch you.

When you go through immigration, there is usually an entry in the country's computer system recording your entrance, and generally your exit. Countries have varying degrees of success catching people residing illegally in their borders, and some countries do have sizable populations of people living there after overstaying, but in general, you can be caught in a number of different ways:

• When you eventually try to leave, they can match up your exit to your entry. You may be fined or face other consequences depending on local laws.
• If you have contact with government officials, they may require proof of your legal status in the country, especially if you are stopped by the police.
• Common actions like getting a hotel room, job, apartment, or bank account may involve showing proof of legal status and/or a report to various government agencies.
• There may be immigration or police checkpoints inside a country.
• The authorities may raid places where they believe immigration violators are present, such as an employer alleged to be hiring workers illegally.
• Someone may report you to the authorities.

Some countries collect fingerprints, photographs, iris scans, or other biometric identifiers at the border to help catch criminals and immigration violators.

At a minimum, there will generally be a record in their database of your overstay, and you may be refused entry if you try to come back to that country in the future, or be required to apply for a visa before you can be readmitted.

In addition to the two excellent answers so far, let me point out that some nations will let some foreigners in with just an identity card which has no room for any stamps or visa. That happens if there are friendly relations and a low perceived risk of overstay. That is discrimination, but nations are allowed to do it ...

For citizens of the US or EU, getting a visa is often just a tedious formality if it is necessary at all. For citizens of Middle Eastern or African nations, getting a visa can be much more difficult.

Nations which are popular tourist destinations will have to decide if they want to make it easy to visit or if they hold out for reciprocity in the visa requirements, compare this recent question.