Where can I find GPS-device-compatible maps of Georgia?

The Openstreetmap wiki has an overview of downloadable maps for Garmin devices. If you're interested in hiking, you may want to look at their hiking maps page; those services may include more information of interest to hikers and some include contours as well, but most are not (easily) downloadable for offline use on Garmin devices. In general I used to recommend Garmin Openstreetmap, but this service is often down. These days I use userbeam for downloadable maps (but right now I get a certificate error here). When it works, in the drop-down list, just select Europe -> Georgia (they classify Georgia as Europe, even though it's mostly Asia). Otherwise check out other sources on the linked overview.

Note that it does not cover topographic information, so it is not enough for hiking. For Georgia, I would print out some old soviet maps, which probably cover Georgia pretty well. If you're lucky enough to be in a highly frequented area, there are probably commercial topographic maps as well. I've also had some luck on downloading Soviet topo maps to my Garmin device, but they were very slow to load, they're (naturally) not routable, and therefore not great as a sole solution.

The Garmen GPS maps do not show detail in the regions such as Mtskheta where Kazbegi is located. I used and still use the free app maps.me in Mtskheta, Kakheti, Shida Kartli, Samtskhe-Javakheti, Imereti, Guria, and Adjara. To get this, you have to first download the app to your iPhone, then the Georgia specific map. It works offline, shows your location, maps directions, and has the pin points for some sites of interest.

Quite good maps you can download http://garmin.opentopomap.org/ If you want to get some GPS coordinates of Georgia Svaneti area, some trek coordinates for GPS device: http://www.zmidloch.com/outdoor-blog/svaneti-trek-georgian-caucasus/