You’ve set up perfectly, but for some reason the first flight landed in another part of the field. And now that the birds are down, subsequent flights are sure to follow suit. If the birds went straight in, they were likely not avoiding you—they simply decided they wanted to land somewhere else (this happens much more often with geese than with ducks).
What you need to do in this situation is immediately get the birds back up. To do this, try to quickly sneak up on them and get a shot off, but be careful not to harass already settled birds. Once you have the birds in the air, leave a flag or coyote decoy where they had landed and get back to your blind.
That should work, but if the next flight also doesn’t come to your spread, you have no choice but to pick up and move. Grab your blind and as many decoys as you can carry and re-establish where the birds want to be. Act quickly so you’re in place before another flight arrives. Even if it’s a pretty rough set-up, at least you’ll be giving yourself a chance.
If the birds have chosen another field altogether, however, it’s probably a scouting issue, underscoring the importance of watching a field more than once before choosing to hunt there, as well as the importance of putting the birds to bed the night before your hunt. In this case, you’re best to pack up, get permission to access the field the birds have chosen, and try again that evening or the following morning.