How to remove bees from window frame

Just wondering if anybody has had this problem and if so how did they solve it. The problem area is at the back of the house which gets lots of sun most of the day so is nice and warm. The last week or so we have noticed lots of bees around the window sills. It got so bad we could not have the windows open as I do not like things flying around me although they do not seem agressive. When we investigated further we found that the bees were getting in and out of the vents which are there for drainage . We do not want to kill the bees we just do not want them living in our window frames. We have bee houses in other parts of the garden which are lovely to observe and plant lots of things to give them nectar etc. There are four windows affected at the moment and I guesss each one must have a queen in there. Does anybody know how long it would be before all the bees left there of their own accord and how we might prevent them doing the same next year. Any helpful comments or experiences welcomed.