I'm 34 weeks and still going to my box. I have to adapt a lot of things, but there are bumpers and things at the box that no other gym in my area would have. I'm still lifting, though obviously lighter, but for a 75# shoulder press or something I really want the opportunity to drop the bar if I want to. You can't do that at most Y's.
As far as scaling and such goes, I'm still doing (mostly) the box workouts, but doing pregnant burpees, singles instead of double unders, and so on. I still do the full workout, most of the time. Our box tends to do a strength component and then a WOD, so for me that's really helpful.
I agree with the PP that it's really just a personal choice for you. I don't have any experience with IVF, but I do know that I was much more cautious at the beginning of the pregnancy than I am now in the sense that I now know better what are normal aches and pains, what are round ligament issues, what Braxton Hicks feel like, and so on. As you get more comfortable with your workouts you may want to go back to the box, so that's always an option.