Schengen Visa Annulment and Subsequent Removal

Your Schengen was annulled and you were removed from the Schengen zone. You didn't give the reason for this, but there are only a few, four in fact. These are listed in the Visa Code, Article 34 and Annex VI

1. A visa shall be annulled where it becomes evident that the conditions for issuing it were not met at the time when it was issued, in particular if there are serious grounds for believing that the visa was fraudulently obtained. A visa shall in principle be annulled by the competent authorities of the Member State which issued it. A visa may be annulled by the competent authorities of another Member State, in which case the authorities of the Member State that issued the visa shall be informed of such annulment.
2. A visa shall be revoked where it becomes evident that the conditions for issuing it are no longer met. A visa shall in principle be revoked by the competent authorities of the Member State which issued it. A visa may be revoked by the competent authorities of another Member State, in which case the authorities of the Member State that issued the visa shall be informed of such revocation.
3. A visa may be revoked at the request of the visa holder. The competent authorities of the Member States that issued the visa shall be informed of such revocation.
4. Failure of the visa holder to produce, at the border, one or more of the supporting documents referred to in Article 14(3), shall not automatically lead to a decision to annul or revoke the visa.

Based upon the scant information you provided, it can be assumed they used the first reason "A visa shall be annulled where it becomes evident that the conditions for issuing it were not met at the time when it was issued, in particular if there are serious grounds for believing that the visa was fraudulently obtained." They will use this reason if you are not following through on your itinerary or cannot produce evidence that you will follow through on your itinerary.

Further to the comments given by Flimzy (to whom thanks), German border officials do not select someone randomly and remove them on a whim. To the contrary, they follow the rules with relentless precision and the rules call for them to brief you on the grounds for your removal and to be reasonably sure that you understand them.

7. A visa holder whose visa has been annulled or revoked shall have the right to appeal, unless the visa was revoked at his request in accordance with paragraph 3. Appeals shall be conducted against the Member State that has taken the decision on the annulment or revocation and in accordance with the national law of that Member State. Member States shall provide applicants with information regarding the procedure to be followed in the event of an appeal, as specified in Annex VI.

What can I do to make clear it from my record?

When you were removed, they explained the reasons to you and gave you some paperwork. This will specify the options you have for appealing the annulment of your visa. You can read the paperwork directly or have a translation made for you. You will then have all of the information you need to launch an appeal. If they did not give you paperwork about the appeal process, it means your removal was on grounds that cannot be appealed.

That answers your question. As a strategic tip, you can consider that if they got you on something and went through the removal procedures their grounds are probably pretty solid and winning an appeal against removal will be a lengthy and costly process with little chance of success. This would be especially true if you do not understand why you were removed. What if they are right? So I'll provide another option...

A more convincing way to clear the record is to get another Schengen and perform against it in such a way that it is not annulled. This will allow you to claim that your removal was a one-off event which is unlikely to be repeated.

Adding 7 Oct 2015

Commentary by "Relaxed" (to whom thanks) has pointed out that...

It might be useful to highlight the fact that a visa should be annulled (as opposed to merely revoked) “if the conditions for issuing it were not met at the time when it was issued”, meaning that the border guards believe the OP did not merely change his plans after the fact but simply lied about his intents to get the visa.