Robbie Legg put the Sarnians ahead after a bright start but Serge Makofo equalised before half-time.
Second-half strikes from Chace O'Neill and Lee Hall and another from Makofo ensured GFC's miserable away run continued.
'We never go into a game not trying to win, but there is no point talking about the play-offs,' Vance said.
'We have blown that opportunity unfortunately.
'The poor run we've had recently, and other teams taking advantage of that, has pretty much finished off our challenge.'
Full match report in Mondays Guernsey Press.
