Preheat oven to 400 F. Cut potatoes in half, rub with olive oil. Place onto a large baking sheet. Bake for 30-35 minutes. Let cool slightly.
Scoop the middles out of each potato and set onto a plate. Place the potato filling into a medium bowl. Mix with mayonnaise, mustard, salt, and pepper. Spoon filling back into each potato shell. Refrigerate until serving.