(later) Here's the kind of conversation that I'm thinking about:
Baha'i: All religions are one.
Christian, Buddhist, Muslim, or follower of some other religion: But these beliefs and practices of yours are incompatible with mine.
Baha'i: No they aren't because ...
or
Baha'i: That's because you aren't really following the teachings of your Manifestation, like we are.
Either way, the message is "I'm not interested in what you think about your religion," which people can feel even if they can't see through all the smoke and mirrors around it. Baha'is aren't really much different from other people that way. What makes it look worse to people sometimes might be thinking that "all religions are one" means "all religions are equal." It might be confusing and frustrating to people sometimes to think that Baha'is are saying that all religions are equal, and see them acting like they are infallible interpreters of the scriptures of all religions, at the same time. According to their own teachings, that can't be true, because none of them are even infallible interpreters of their own scriptures.