You don’t really need the carbon filter if you’re not worried about smell. You can always hook it up later if need be. But like
@gmslave said, exhaust and fresh air is important so you should keep the fan running. Don’t sacrifice air exchange for a little more space.