Is my Dutch driver's license a valid form of identification in Belgium or France?

In France, there is no notion of a legally valid form of identification that should be presented to the police when asked and certainly no obligation to carry any given document. French citizens are perfectly entitled to live their whole lives without even holding (let alone carrying) a passport or ID card (it would not be very practical but it's not illegal per se). Foreign residents (and especially third-country nationals) are strongly encouraged to carry their residence permit and can face a lot of hassle (up to 16 hours of detention) if they don't have it with them during a police check but even in this case, it's not an offense not to carry it.

What matters is your status and if you can't document it, the police can keep you for some time to establish it. Obivously, this should almost never happen if you have a national ID card or EU passport on you but you won't find any law listing what is or is not a valid means of identification and official advice carefully dances around the issue. That's completely different from the situation in the Netherlands where not carrying the relevant document is an offense in itself, punishable by a fine.

As an EU citizen, you have a very strong right to be in France and you only need to convince the police that detaining you would be a waste of time because you're not on any list of wanted persons (which would ostensibly be the main justification for increased checks after recent events) and cannot be removed. That's why a Dutch driving license and a passport copy should be enough to satisfy any reasonable police officer during a random police check.

At the border, the situation is a bit more ambiguous. The police might be tempted to simply ask you to turn back, which is legally questionable but still happens and would obviously be a major inconvenience. For all the reasons I gave above, I don't think this would be particularly likely but I can't categorically rule it out either.