Extremely slow availability appearance on round robin with large pool of event hosts

We have a group of about 40 calendars where a round robin event has to grab availability and who’s next to get an intro meeting with a person using our experience. 

As soon as the person views the monthly calendar, it takes 20-30 seconds to load availability. If they go to the next month, it again takes 20-30 seconds to load the calendar. 

We’re tied to Outlook calendars and I imagine that is the reason for the extreme slowness. Are there ways to configure Calendly or our integration with Outlook to speed this up?