I've worked with a few user experience designers who haven't been exposed to major cultures or cities different from their own, despite being in proximity with one. Others don't even want to touch or visit a different place, domestic or international. In my industry experience, I've seen the constant of Silicon Valley self-absorption and its resident techies: a select few users who've been user tested over and over again or a population who only represent one's primary geographical locale and likely nowhere else.
These experiences outside of our own individual mindsets can provide SO much inspiration for our work and stretch our horizons from our mundane routine of everyday life to an expanded perspective. If you aren't exposed to this, you aren't truly understanding the human experience needed to do your job well, which is namely to study and design for humans (obviously super simplified). It's not okay to only design for people you know or who are just like you.
I'm hoping this doesn't sound so privileged. I understand that travel is expensive and it's hard to make time for being away from the everyday - your job, your family, your routine. There are so many priorities in life that are fighting for our attention. But perhaps find a way, big or small, to inject that world of culture into your work, whatever it means to you, whether that be an interestingly diverse culture or a new city. It's super important. Travel and exploring new cultures breeds more understanding. It doesn't have to be frequent. It doesn't even have to be intense. Find ways to explore those other worlds, and then pull from those to influence your design for good.