Previously denied entry from Canada to USA. Does that technically count as setting foot "on US soil"?

15 years ago, we decided on a whim to get lunch in the USA as we were passing near the frontier on a road trip in Canada. We did not know the exact requirements, and as we lacked an address of destination and US currency we were refused entry by the US officer. After filling some forms, the officer showed us the roundabout on the US side, told us to go around it and go back to Canada. I remember him insisting "you have never been to the US", I found it strange, as de facto we were on US soil, even so for a few minutes.

Two years after that we immigrated to Canada, we've been citizen of this country for 10 years or so. My question is not about the risk of being scrutinized or denied entry when I go back there –I found plenty of answers here about it– but specifically what I should answer if a border officer ask me "Have you ever put foot on US soil?"

I feel that if answer "yes" I might get accused of lying (as contradicting the imperious statement of the previous officer) but if say "no" I'll be accused of the same thing as I, indeed, in reality, physically been to this country before.

(I've considered explaining the situation, but my experience of talking to these border control agents is that they don't like complicated stories, that make them go paranoid.)