Yes, 112 works quite well in Germany, but there are differences to other countries.

There are two main emergency numbers in Germany.

The Police (Polizei)

To call the police, dial 110. It's usually answered by a switchboard that is in the area of where you are, by someone who is with the police. In large cities they have their own switchboard. You don't need to say I'm in Berlin. You can just say I'm in Kreuzberg. But if you are calling while you're driving on the Autobahn somewhere between towns, you'll end up in a regional one, and they will ask between what towns, which road, what section and direction of the road and so on. (Only call hands-free while driving, calling and driving is not allowed in Germany and gets expensive).

When not to call

Don't call 110 if you want to just talk to the police without an emergency. If your wallet got stolen and you want to report that, go to the police instead, or call the normal local phone number of a police station. If you think your car got stolen and you are in Hamburg or Berlin, also don't call 110, but google for Umsetzung first, as it's possible your car was in the way and the Police had it moved to a different location.

The emergency hotline

The number for 112 on the other hand is for both medical emergencies (Krankenwagen) and for the fire fighters, called Feuerwehr in Germany. So if you need an ambulance, you call 112. If there is a fire, you call 112. If you see oil on the street, smell gas and think there is a leak, your basement is full of water, or if you witness a car accident, you call 112. That number has its own regional switchboard. They are located in a regional command center that's also usually where units are dispatched from. Here's a bit of German information about the one in Hannover. They also dispatch the Katastrophenschutz, which helps with floods and things like that.

Note that for the car accident, the 112 dispatch will notify the police, so always call 112 first when people are injured, even if there is road traffic or crime involved.

When not to call

Don't call 112 if you want to plan an event (like a company football game or a publicly accessible party) where the law requires you to have paramedics present or the firefighter to be made aware first.

How to call

Both of these emergency numbers have a protocol you should follow when you call. Those are referred to and taught to kids in school as the five W-Questions, because all of them start with a W in German.

• Where did it happen?
• What happened?
• How many people are involved/injured?
• What kind of damage/injury?
• Who is calling?

Never hang up when you call them. Always wait until they end the call, because they might have additional questions.

It's possible that you get routed to the police if you call 112 if there is no local firefighters dispatch available. The operator will identify clearly where you ended up.

There is also a ton of information in German in the Wikipedia page about Notruf.

Note that the emergency hotline can be busy. Depending on where you are, they might only have one person operating the phone. In Berlin it frequently happens that at 110 you first reach an on hold queue before someone talks to you. So please only call when there really is an emergency.

A similar looking number is the 115. It's not an emergency number, but it has the same format. It's not special in any kind, and in fact is just a local call. It connects you to the municipal government's service hotline, so you can make appointments for registering a residential address in a new town or things like that. It's relevant for expats, but not really if you travel.

Speaking as a (retired) mobile network planner, "dialling" 112 on a mobile with or without a SIM should not actually "dial". It invokes the emergency communications service (which is independent of normal call handling). (I may have used the incorrect term for the service, I would have to go back to my standards to check.)

In other countries, e.g. Australia, where I live, phones are required by law to recognise the local emergency number "000" and invoke the emergency communications service, and similar rules are in place in most jurisdictions, although 112 is always recognised (on GSM phones) and I implemented it on CDMA.

What actually happens when you invoke the emergency communications service is dependent on the local administration.

(More a comment than an answer):

I've looked at the German Wikipedia entry wrt. the emergency numbers:

It's enlightening that 112 is both the European emergency number (since 1991) and in Germany it is also the emergency number for the fire brigade and the ambulance, historically (since 1973).

The police historically has used 110 (as explained in other answers).

I would assume therefore that, in Germany, 112 gets a lot more fire+medical traffic than the police one and is a possible explanation for the seeming incompetence of the operator - many locals would call 112 only for the fire and ambulance, and it's indistinguishable whether you intended the (historic) 112 or the international 112 :-)

Quite some other European countries have completely disjoint police/fire/medical emergency numbers vs. the international 112.

As far as I remember (but I can't find a source right now), this lack of interconnection between the emergency phone systems of the fire brigade and police is intentional. Some reasons for this are:

• It can make it easier for people to call for medical help without having the police know about it (for example, if illicit substances are involved).

• It frees up the operator and telephone lines, by not having to handle the transfer.

It does not indicate a lack of training or motivation on the operator's part.

This policy seems to violate (albeit slightly) EU regulations, similarly to the requirement to have a SIM card when calling 112.