Does applying for a UK Visitor visa from a country where I don't have a residence permit reduce my chances of getting visa?

I am a non-EU citizen, having a student residence permit from Czech Republic. At the moment I am doing an internship in Switzerland (CERN). Since the internship is less then 90 days, I can stay in Switzerland with my Czech permit and don't need another visa/permit.

I have been invited to a job interview in the UK and need to get a UK Visitor visa to go there. The company assigned me a visa application consultant.

I want to apply from a visa centre in Switzerland. However, the consultant insists that I should apply either from a country of my citizenship or a country where I have a residence permit. Otherwise, they believe that my application is very likely to be rejected (just because I don't have a residence permit/citizenship here). I don't want to do that, since that would require me to go to another country in person, and also be stuck there for at least a week, since I must leave my passport with my application.

As far as I understand, legally it is totally fine to apply for a visitor visa in a country where I don't have residence/citizenship. However, the consultant points out that it is strongly recommended to have residence and there is some document ("UK Visas & Immigration; Visitor: supporting documents guide") where the following is listed as highly recommended to be provided with visa application: "Confirmation of legal residence, if you are not a national of the country in which you are applying or your right to reside there is not included in your passport".

I don't have that, therefore they believe my application is likely to be rejected.

Are they correct here? Is it really a problem to apply from a foreign country?