Strangely enough, this has come up in game.
If you have a temporary Overseer, I think you should also lose whatever temporary Resources are tied to that overseer (so Falleniel would lose 1 Temp Overseer and 1 Temp Spies)

Re: choosing which Resource he'd lose - Usually, I roll a die for the 'randomness' of losing temporary resources. However, if the player had made a point of tieing the overseer to a specific resource, the player would lose the resource 'tied' to the Overseer.

Does that make sense? It's probably too late for me to be answering questions.

Makes sense. I don't like it, but it makes sense. It fits in with the game's theme of it's easier to destroy than to build (my least favorite part of the game).