I've always wondered how people who advocate for 
  more open borders justify their position.

To me, it seems like the default position, as long as one believes in liberty and, obviously, doesn't hold essentialist views about culture.

Mind you, my position isn't "open all borders now, and damn the consequences!" It's a goal for the world to work towards, not something that we can do without accounting for economics, law, and xenophobia.