The Unsullied, Men of the Night's Watch, and the City Watch are just three types of soldier in Westeros and beyond, yet they make up a microscopic proportion of the amount of soldiers in the realm. Those particular three groups are unlikely to meet anytime soon (if ever), though the men beneath the helmets tend to be the same. So when you see an unsullied warrior wiping out a slaver, you could see the same man a minute later being cut down by a white walker, then just as quickly guarding Cersei during a tense courtroom scene.