Everyone's different. I share a bowl with my wife every night while we are winding down before bed. Haven't had any negative side effects.
When I was in my twenties, I was a much heavier smoker. I was deeply unhappy with my circumstances, and used it as a way to shut my brain off after work or on weekends. The worst side effect there was hanging out with people I shouldn't have, and spending too much money on it. Now, it's cheap and everywhere.