Is it better to ask a question in English or Japanese if you are expecting a reply in English?

I would still suggest asking the question in Japanese is better. It at least shows you're trying and in most cultures this is appreciated.

If the people didn't answer your question it might be that they didn't understand what you were saying. At that point, it would seem OK to switch to English to see if they understand. At least you've demonstrated a small effort.

Even better of course is to try and learn more phrases in Japanese so you don't have to ask whether they speak English.

Well first, we assume you are pronouncing it correctly when you say it. So in that case, there are a few options.

• you could be saying it clearly, and they themselves don't speak English, so are embarrassed to admit this.
• they may speak English, but are embarrassed to try their broken English, and at the same time are worried your Japanese sounds poor, and as the 'hosts' of you in their country, do not want to embarrass their guest.
• they are deaf? :)

My first thought when reading your title was 'err, English' - as by speaking in English, it's immediately clear what your purpose is, and there's no false impression given by your attempting to speak Japanese. However, when I noticed you were in Japan (body of question), my answer changes to definitely asking in Japanese first.

It's important to respect the culture and country of the place you're visiting, and not assume everyone speaks your own language (be it English, Japanese, German, Afrikaans or whatever). So it's helpful to always learn a few phrases, and one very handy one is that 'do you speak English?' or 'I only speak English' phrase - even if they don't, they may be able to find someone who does speak English. And if they do, they will at least respect your attempts to use their language.

Failing that, you can resort to the flailing arms and hand signals method of communication ;)

"Do you speak English" is different from other questions. "Which way to the train station, please?" might be best in English, especially if you don't know how to understand the answer in the local language. But "Do you speak English?" in English carries with it that you can only speak English and nothing else. It wouldn't be a good first sentence of a conversation - though if you speak nothing else, what choice do you have?