I've gone through this before but:
My first criteria is dry roads. That means if it's below freezing, the weather has to have been dry for a few days prior, to decrease the chance of black ice from frost/ground water.
After that, I ride down to about 25F with gear. Between 15 and 25, I generally don't ride, cuz it's not as fun and I'm not setting a record. Below 15, I'll ride in the hopes of setting a new low temp riding record (currently it's 12F).