Hey Jericho, without using a carbon filter they will smell it haha.
A friend of mine had a problem with his fuse box and the carbon filter was off for like 1 day, the whole street smelled like weed (in flowering stage) haha.
You should do something about it ;)
Having a carbon filter is also awesome to keep the temperature at a better range :)
Good luck growing :)