Sure. Then you run the election and if anybody else in the area wants to run in the same party as the person who left then the party can put them on the ballot. If not, then no candidate for that party. Don’t drop somebody in from elsewhere. Pretty simple. Just pick a time period that a person needs to have lived in a riding (say, one full election cycle) before running to represent the area.

If the person wins the election and moves (other than to government housing in Ottawa so they can do their job) then they give up their seat.

The only reason the seat is vacant is because they wanted to give PP another chance. Without that, they still have their MP.