Hey there! You hit the nail on the head - this is almost certainly a timezone issue and a misunderstanding on the end of the invitees (especially given that it has been two isolated incidents).
Calendly intelligently detects the time zone of your invitees based on their device settings. When they visit your scheduling page, your availability will be displayed in their time zone. If it is your goal to only have in-person meetings, then you will want to lock the way time zones appear to your invitees. You can learn more about Calendly's time zone settings here.
In addition to this - the timezone displayed when the invitee books will show up in a few places. First, they’ll get the confirmation page - this will display in the timezone that was detected (or manually selected) on your booking page. Next, they’ll see their calendar invitation or email confirmation - this is where things get a little sticky. Their calendars/emails will display the time that their device is in, in most cases, but the email will state the time they booked in on your booking page, so sometimes there can be some differences and confusion. I would assure the invitee of the times you see on your “scheduled events tab” (where you can see what timezone the invitee booked in vs. your own timezone for the event) and ask that they double check their device settings etc. If you run into further issues, let us know!
