Checkout Allowing Users To Click Back

We have noticed customers have placed an order then been able to delete it and change the order. We didn't know how. Today it happened again and we worked out that users are going through to the payment gateway and then clicking the back button several times, this brings the user back to checkout on site and allows them to modify their order.

The client this morning swapped his items for another item and then paid again, a few days from now he will call going mad that he has paid twice.

How can we stop this from happening ?

I can assume that the issue is payment method related. For example, customer is redirected back to the store before transaction status is received by the store (usually callbacks are used to do it). In this case customer is redirected back to checkout and cart items are not deleted