In Europe, where is a water border with visible markers, which can be crossed by a tourist in a small rented boat, without a passport check?

Probably your best bet is, to find waterways - particularly canals - that go across borders, not along:

• Across, not along, because European rivers tend to have a lot of water traffic. I don't remember ever seeing a river or similar along a border that had markers down the middle - it'd be an unnecessary obstacle to boats - but waterways across borders will usually have a friendly welcome sign at the border point.
• Canals are more likely than rivers because for military reasons, rivers in Europe near borders tended to become the borders, more often than crossing the border... But canals started to be built in more peaceful times.
• Within an open border zone like Shengen or mainland Britain, otherwise it's likely to be closed with border-crossers funnelled into a major official crossing (usually a main road)

Here's one example (assuming you count it as a border) - the Llangollen Canal which crosses the border between England and Wales:

• It's a completely open border (two countries within one nation state) so you can cross it at leisure. If you don't think it's a real border, don't say so where a Welsh person can hear you :-)

• It has cheerful signs marking the border:

  

• It's extremely scenic and unique. You might not realise from that first photo, that the border crossing is actually high up on a historic aquaduct. Two pics from that Wikipedia article:

  

• While you're there, on the Welsh side of the border further along the same canal, there's an even more spectacular aqueduct, Pontcysyllte (pics from Wikipedia and Reddit).

  

• It's a popular desitination for all sorts of boating holidays. You can hire a narrow boat with an engine and enough space for a family to sleep and cook meals, and spend a few days cruising the canal, or you can paddle across in a canoe, or one of those weird standing canoe things (don't fall off!) called SUPs apparently. Pics from Tripadvisor and SUPnorth...

  

I upset the OP with my Dover answer so here is a bit more serious take on the matter, hopefully he OP will forgive me now.

This is Rudawka - Lesnaja (map) tourist water border crossing on Poland - Belarus border open from May to October. It was opened to allow uninterrupted paddling between the two countries. Having said that you have to arrange the crossing in advance with the appropriate office (e.g. Republic of Belarus consulate in Bia?ystok). You can expect to get detailed information at tourist information offices in the area.

The crossing is located at the Kurzyniec Lock, one of many locks on the Augustów Canal, a cross-border canal built in the 19th century.

This is a different lock on the Augustów Canal. Gives you the idea of the fun ride you can have there. The canal is 63 miles long and features 18 locks. You can paddle along the whole length of the canal.

The river border between Belgium and the Netherlands, in Dutch 'de Maas' and in French called 'La Meuse'.

The stretch measured in a straight line is about 19.5 mile, about 31.5 km, (Google maps) but because of the nature of the river much longer by water, and also going by road will take much longer. That is between Maasbracht, the Netherlands - Kessenich, Belgium and Lanaken, Belgium - Maastricht, the Netherlands.

Ferry crossing 'de Maas' at Grubbenvorst and Velden
Photo released to public domain, by Gouwenaar.

There has been a recent border adjustment as the river had changed its bedding but the official border was still on its fixed paper line. A part of the border on the Dutch side of the river was officially still Belgium and while the people just walked over the border, Belgium police and other officials had to apply for permission to pass the Netherlands into the part of Belgium. You can find more about it in English language newpapers and online versions, as far away as NY Times, this Irish one comes with several photos.

As you can read in those news snippets, it is easy and allowed to cross the river by small craft, no border officials and also no or not much in the way of border markings, depending on where of the river you happen to cross. There might be some ancient markers from where there has been an official border crossing over the centuries, it is only in the last 40 years that there are no border checks between Belgium and the Netherlands, I am not even sure how long myself.

The border crossing of the river Rhine (Rijn) from Germany into the Netherlands takes about 4.67 miles, about 7.5 km (measuring curtecy of Google maps) and due to the nature of the river, pleasure boating is less encouraged, but still possible and allowed. Spijk in the Netherlands is along one side the river as is Tolkamer, Kleve in Germany on the other. Border checks did continue here till the start of the Schengen zone allowed for check-free crossings. If you go to 51.846848, 6.154735 on google maps and use street view you can see the river, as it was the moment their camera vehicle came past.

Rhine with groynes near Tolkamer, Netherlands.
By Gouwenaar (Own work) [Public domain], via Wikimedia Commons

There is a roughly 2.5 mile, 4 km stretch of border north of there, about as far north as you can go along that border. Due to the laws in Germany, or at least to the way Google streetview implements it, I can not give you a few of the water at the actual border, this is the nearest I can get in the Netherlands: 53.183672, 7.203187 The town is called Bad Nieuweschans, the Netherlands, on the other side of the border you can find Bunde, Germany. The water there looks like man made, but it might be a canalized bit of river as well. Again, as far as I can see no rules against pleasure boats, but I am not sure, not having been there. I know it is accessible by car/foot as my brother has visited the area once.

That is just three on the border of the Netherlands, I am sure you can go around other Schengen borders to find more. Use a 'walk around' option with the online map, wherever possible, to check out the nature of the borders.

I agree with the other answer, if you want to see signs of the border in the water, better check where the water crosses the border rather than being the border. And the more recent the crossing was guarded, the more likely you will find signs of guarding. And the more defended the border was, the more likely you will find proof in the river even after years.

In that respect, you might want to check out where used to be the border between the two Germanies. No border anymore, when it became a border between two political blocks it was one nation with all kind of waterways crossing the line, and there is no reason to take out all signs of having been a border unless it is big enough for serious boat traffic.
But I am not sure it fits your requirements as you do mention it being an actual national border.

How about this one in Dover.

• water? check!
• boats? check!
• markers? check!
• guns? check!
• searchlights? if you are hiding in one of the lorries there for sure, check!