Do I need a bus/train/plane ticket out of Thailand in order to get a Visa-on-Arrival?

From ThaiEmbassy.com (not the official website of the Thai embassy):

They must have a confirmed return ticket to show that they are flying out of Thailand within 30 or 15 days of entry, as appropriate. Open tickets do not qualify. Travelling overland out of Thailand by train, bus, etc to Cambodia, Laos, Malaysia (including en route to Singapore), Myanmar, etc is not accepted as proof of exiting Thailand. You may be asked to show your flight ticket on entering Thailand. If you do not possess a flight ticket to show you will be exiting Thailand within 30 or 15 days of entry you will be most likely to be refused entry.

So, technically you need a onward flight ticket (not necessarily a return to where you came from).

Now, someone may come along and say they don't really check for those details, and they probably don't unless you make them suspicious but technically they can refuse to give you a visa.

However, your profile says your from Canada, if you hold a Canadian passport then you qualify for the tourist visa exemption which has different rules:

Foreigners entering Thailand by any means under the Tourist Visa Exemption category are required at the port of entry to have proof of onward travel (confirmed air, train, bus or boat tickets) to leave Thailand within 30 days of the arrival date (otherwise a tourist visa must be obtained).

It's also worth noting this:

... passport holders from 41 countries and 1 special administrative region - Hong Kong SAR - are not required to obtain a visa when entering Thailand for tourism purposes and will be permitted to stay in Thailand for a period not exceeding 30 days on each visit. If such foreigners enter Thailand at immigration checkpoints which border neighbouring countries (overland crossing), they will be allowed to stay for 15 days each time.

So factor that into your plans and it does look like you should have some proof that you're going to leave within the 30 day window (your flight out will work for your reentry into Thailand but you'll only get 15 days if you come in overland.)

Now, in my experience I've never been asked for proof of onward travel when entering Thailand (at Bangkok airport). Your experience may be different.

It sounds like the informal policy is changing.

As Wikitravel mentions:

Proof of onward transit :- long happily ignored by Thai immigration, has been known to be strictly applied in some instances (Indian passport holders beware). The requirement is for an international flight itinerary - NOT train, ferry, or other departure type. Airlines, who have to pay for your return flight if Thai immigration doesn't let you into the country, also check this and often will not let you board your flight for Thailand without it.) A print-out of an international e-ticket on a budget airline is sufficient to convince the enforcers, but those planning on continuing by land may have to get a little creative. Buying a fully refundable ticket and getting it refunded once in Thailand is also an option. Land crossings, on the other hand, are a very straightforward process and proof of onward journey is generally not required (Indian passport holders beware again... or anyone, if the border officials simply decide to uphold the bureaucracy).

Since your ticket out is AFTER the 30 days of the visa, it'll raise a flag - they'll wonder if you're planning on overstaying. You could show proof of accommodation booked in other places if you have it, or other transport booked, but there's no guaranteed.

So long story short, you might be ok, but your best bet if you want to be sure, is to use skyscanner or kayak, find the cheapest possible or refundable flight out of Thailand to anywhere, book it and use that as your 'proof' that you're leaving. Then either dispose of it or get it refunded.

From the Thai Embassy:

You "must have a confirmed return ticket to show that they are flying out of Thailand within 30 or 15 days of entry, as appropriate. Open tickets do not qualify. Travelling overland out of Thailand by train, bus, etc to Cambodia, Laos, Malaysia (including en route to Singapore), Myanmar, etc is not accepted as proof of exiting Thailand. You may be asked to show your flight ticket on entering Thailand. If you do not possess a flight ticket to show you will be exiting Thailand within 30 or 15 days of entry you will be most likely to be refused entry."

I'm going to assume you're a citizen of Sweden since you're flying from Sweden. The tourist visa exemption is only available for citizens of 48 countries. Sweden is one of them.

By Thailand's rules, your 4 June flight is not adequate proof of onward travel. Here's why:

Itinerary:

18 March: Arrive THA via plane

1 April: Depart THA, travel to LAO, KHM, and VNM, means unknown.

4 June (?): Arrive THA

4 June: Depart THA via plane

18 March to 4 June is 78 days till you leave THA, as far as you can prove. The max you can stay in this situation is 30 days.

Recommendation:

Fly from THA to wherever you plan to go next, that way you have proof of onward travel. Flights on budget airlines in Southeast Asia are relatively inexpensive.

As stated elsewhere, overland means are not acceptable proof.

I can also state that I've entered Thailand multiple times and have never been asked for proof of onward travel. Granted it may already be in their computer system and I've always had proof of onward travel. I wouldn't risk messing with Thai immigration.