Bug/Issue Server has closed the connection-farmhands kicked out of game

In multiple sessions, after joining another person's farm, I and the two other people will get kicked out to the menu with the note "server has closed the connection." The host hasn't had any issues. Usually, we will get to play for 5-10 ish min before we get kicked, but after the first time it goes down to one minute or less, and we can usually only rejoin if a new co-op code is used.