Can I wear all types of military surplus clothing in Germany?

There is indeed some statute suggesting something like that, namely § 132a StGB (which means “paragraph 132a” in the penal code):

(1) Wer unbefugt […] inländische oder ausländische Uniformen, Amtskleidungen oder Amtsabzeichen trägt,

wird mit Freiheitsstrafe bis zu einem Jahr oder mit Geldstrafe bestraft.

which could be translated as “A person who without authorization […] wears local or foreign uniforms, official clothes or official signs will be punished with up to one year imprisonment or a fine”. This sounds like the type of rules DJClayworth was talking about in the comments and is probably the source of the rumor you heard. That said I am not a lawyer and I have no idea how strongly this is enforced, how it has been interpreted by the courts or what precisely is forbidden or not.

Some Internet forums (which of course are not particularly authoritative either) suggest that not wearing a full uniform or making sure some part of it like regimental or rank signs are missing would be enough to avoid falling foul of the law. I would therefore not assume that the fact the army is selling its surplus, if it is, necessarily means it's legal to wear a full uniform. Conceivably, it could also simply be a catch-all text to give legal cover to the police when someone wears some uniform with an intention to deceive, I really have no idea.

More on this in Wikipedia

There is also an entirely different paragraph forbidding signs associated with “unconstitutional organizations” which covers symbols associated with the Nazi party and, presumably, many WW2 uniforms.

This one is covered here and also in English

No, it is not illegal, but it is still not advisable if not applied very carefully.

The highest German civil court, the Bundesgerichtshof, has defined the rules to §132a, at March 15th, 2011 (4 StR 40/11):

Allerdings ist der Tatbestand des § 132a StGB in beiden Tatvarianten nur erfüllt, wenn es sich bei der jeweiligen Uniform bzw. dem Amtsabzeichen um solche handelt, die auf Grund öffentlichrechtlicher Bestimmungen eingeführt sind1. Amtsabzeichen werden zudem nur dann von der Strafvorschrift erfasst, wenn sie, ohne Bestandteil der Amtskleidung zu sein, an vorschriftsmäßigen Uniformen angebracht sind und den Träger als Inhaber eines bestimmten Amtes kennzeichnen.

The violation of the §132a is fulfilled if the given uniform or signs are defined by public designations. Further signs only violate the law if they are used on the correct uniform and designate the wearer as a holder of certain rights.

Still good to remember, just in case you meet some overeager members of the police. I think you do not intend to wear Nazi symbols, so I won't go into detail here.

But it is still not advisable because military gear in cities (if it is not inconspicuous enough) gives you a bad image because it is too much associated with Nazis or thugs. This can be only countered with antagonistic signals (e.g. wearing gothic or punk style, long hair etc.). Outside the cities it is accepted more easily, but you may still get wary looks.

EDIT: To demonstrate what I mean with "overeager", the highest German court needed to decide in all earnest twice that crossed-out swastikas as symbol of antifaschism are NOT forbidden symbols. Sigh...

The other answers cover the legal part pretty thoroughly. Being German, here is some personal experience: As long as you don't show any sign (country flag, badge, any other affiliation) you are pretty much on the safe side, especially if the cut of the clothing does not resemble uniforms currently in use. I know several people who use old police jackets as motorcycling gear (lettering removed) and older army-style coats.

It is not a very unusual sight to see people in camouflage trousers and they are not associated with being Nazi clothing; it is quite popular among punks and leftists also (well, it was, you don't see them as often anymore).