Any unusual campaign setting has one or two (hopefully more, but one or two will do) very unique monstrosities that express the core flavor of the setting. When crafting a gonzo setting for your old-school campaign, or just an atypical sci-fantasy campaign, the tendency is to want a number of races that especially inspire the ‘WTF!?’ reaction. More familiar races give a sense of familiar themes or enable the DM to play against type for effect, but it is our truly weird races that let us know that this world MEANS BUSINESS. Of course, as soon as you’ve designed your crazy telepathic slug-people with a fondness for zoos, you begin to design their culture and eating habits, and next thing you know, you want to detail them as a playable race. Because it seems unfair, somehow, to invent all this info and then leave it out of the hands of the players to dive into to create their own spin and intrigue and interspecies melodrama, right?
First of all, there are only ever two kinds of creature: those that PC’s can play, and those that they can encounter. From that perspective, you can have any number of weird races you want, since that’s all they are—window dressing. Or combat. Otherwise, they function in a way roughly analogous to human forms of interaction. So that shopkeeper who looks like a Rhino walking upright with a hookah pipe in his hand? Perfectly ordinary, save for that odd cinnamon smell that comes and goes. Remembering that, it’s OK to allow the unusual details and complex cultural mores of the vast majority of species to remain in the dark.
If, however, you find that this leaves much to be desired in your new invented race, which was almost all about lunar ritualism and a completely different set of genders and their interactions, then—tough. Ask yourself: What will these creatures mean to my PCs when they encounter 2d6 of them? Will they just be another shade of lizard people? Or will they have some readily apparent, unique feature that forever marks them in the player’s mind? Will my players have to know this elaborate backstory in order to give a shit? Or will they care immediately based on the situation in front of them?
As for PCs playing monsters, I have increasingly come to realize that, if you want your monstrous races to be at all mysterious and foreign, let them be…well, mysterious and foreign. Don’t let players play them. Otherwise, you fall into trying to preserve the unusualness of your race by attempting to school the player on the race’s complex culture and society and mysterious aims and…just stop. The player will usually violate this stuff anyway. If you have a bunch of players fully committed to playing crazy foreign cultures and bizarro physical needs, then make up a set of charts to roll on (Unusual Ritual Practices, Bizarre Dietary Restrictions, etc.) and let them take things from there. Otherwise, you wind up trying to be a roleplaying coach. This is where the whole idea of monstrous races as classes makes sense. You aren’t playing an Elven Wizard or Thief or whatever. You’re playing an Elf. That does Elf things. Not Thief things, though the two are identical on occasion.
One last bit, about monsters in cities. Now, we all know that Mind Flayers want to conquer the galaxy and the Sahuagin want to sacrifice folks to their shark-god. But really, that Illithid down the street is just a book seller*, even if he specializes in tomes relating psionic powers to Continental Philosophy. He has no one else’s agenda but his own. We craft the ‘racial narrative’ so that, when we see a goblin, we know that he cannot be parlayed with. He must be killed, and this apparently happens all the time, so it’s OK. When we see a Mind Flayer, we know now that this whole adventure has to do with the Plot To Take Over The Galaxy. It is a useful shorthand for a game, even if it has Unfortunate Implications. But if you think about cities in our own time, and even more so for great port cities of the past, they are places to escape the troubling implications of one’s culture and heritage and immerse yourself in the freedoms of uniqueness and anonymity. So let your city monsters, those that walk the streets at least, be individuals before they are anything else. Besides, who knows what that Mind Flayer’s spawnmates used to call him at home when he was a tadpole? Maybe he’d rather live in a little apartment and read a book than hang out with those assholes, thank you very much.
* I understand the peculiarities of the Illithid diet render any sort of peaceful coexistence problematic. So what?