beauty pet peeve

I must always wash my dirty brushes after use. It irks me to store them back in the brush pouch after use if they have not yet been cleaned. Should I be really running out of time that I cannot wash my brushes with cosmetics' detergent, I will at least rinse with water and dab dry. I am extremely cautious because I do not want to promote the growth of bacteria on my brushes.