Is ashwagandha (ginseng) allowed in the UK and US?

First of all, to quote Wikipedia "there is no conclusive clinical evidence that it is effective for treating any ailment" and it's completely unregulated.

USA: Can I travel with non-prescription medicines, or other products such as vitamins and health supplements?

Non-prescription medicines (also known as over-the-counter or OTC medicines), vitamins, herbs, and supplement products fall under the jurisdiction of the Food and Drug Administration (FDA). In general FDA will not object to the personal importation of those products, as long as:

• The traveler is carrying the products in their possession (or in their luggage);

• The amount being carried is an amount reasonably considered for personal use.

How do airports differentiate supplements from drugs? is at interesting on this topic.

The UK is harder but if you review Personal food, plant and animal product imports there's nothing in there that would limit you so the final exemption would apply: any other food product not containing any fresh or processed meat or dairy and with less than 50% of processed egg or fishery products.