Can a non-resident US citizen who has a driving license from a US state drive in another state using his foreign license?

I work at an employer that does motor licensing services in a Canadian province, so I have some experience in this direction.

Generally, with rare exceptions, US states and Canadian provinces do not permit multiple licensing. Licenses usually have to be surrendered to acquire a local license. Residency is generally required to qualify for licensing. Exceptions are most frequently provided for temporary absences, but usually have to be specifically arranged.

Having a foreign license and living in that foreign country, it is in fact your foreign license that is the valid license for driving in North America. It is likely not valid for you to drive in the US on your US license.

In the reverse situation, where you still have your old US license but forgot your foreign one, the safe thing here would be to check with the U.S. state that issued your driver's license and see if they consider it to still be valid (you will need to disclose the possession of a foreign driver's license to get the proper answer to this question). If that license is still valid and in force, even though you have a foreign license, then I see no problem.

Note too that when you renew a driver's license, there are usually questions about residency. For example, here in Saskatchewan, you will have to disclose where you are living when you renew, and this would normally prevent you from renewing unless your circumstances fit the exception criteria. Eligibility to renew is a good benchmark of eligibility to continue using an existing driver's license when you are no longer a resident.

Note that generally, proof of licensing is just about evidence. Your non-US license remains valid, even if you don't have it in your possession - you simply can't easily prove that you have it, and it is legally required that you have it in your possession in case law enforcement asks for it. But merely having one protects you from the effect of many laws requiring proper licensing.