I'll assume it's happening when your lights go out. Personally I wouldn't worry about it, temp swings like that are totally normal. If you feel you absolutely HAVE to level it out, add a space heater that has an internal thermostat that's set to kick on when you're below your target temp.