How to enter from Russia to Abkhazia?

Long story short - no, you can't. Russia states the Abkhazia as an independent state, they have border controls and customs between two "countries". So you'll definitely get a stamp in your passport (you wouldn't get that only if you have a residence permit, maybe).

Official site (in Russian) states, that almost every country in the world didn't support the Abkhazia's independence, so you need a visa to it (exceptions are: Russia, Nicaragua, Tuvalu, South Ossetia and Transnistria), so you probably need a visa too. It wouldn't be glued in your passport, but still you have to get it.

Update: as @martin.koeberl commented, you may got a stamp on the visa, which isn't being glued into your passport, so, maybe, you'll got nothing in it. More information here.

What @VMAtm wrote is correct ( except that "official site" not official. the oficial site is http://abkhazia.travel/en/border (in English)). But the info is correct - you'll receive border crossing stamps on Russian side, and you'll need a visa to enter Abkhazia.

To get the visa: write them email, receive "Entry Permit Letter" email, print & show at the Abkhazian border, but then you need to go to Sukhum to get actual visa (not immediately, anytime before you leave Abkhazia), without which you could have problems leaving. And this is the part that suck the most, because there's not much to see in Sukhum, and the agency that gives the visas does not work during weekends, and it takes 2 hrs to get to Sukhum. While most people just want to visit for one day to see natural beauties, which are mostly located closer to Russian border.