Bug/Issue Friend disconnects in the mines

I started playing with my friend again recently and we've had no connection issues whatsoever.

However when we go to the mines, he disconnects a lot. He'll stop moving or doing anything on my screen, but he can still see me move and doing stuff. And if he tries to go to the next floor or leave the mines he gets an endless loading screen. I think it seems to happen the more monsters there are?

What can we do to fix this? Does anyone else have this problem?