Preheat oven to 400 F. Wash potatoes and poke holes. Rub each potato with olive oil and place on a baking sheet. Bake for 1 hour. Let cool slightly.
When potatoes are cool enough to handle, slice the tops of the potatoes off and scoop out the center. Mash the potato filling with butter, milk, cooked broccoli, salt, pepper and 1/2 cup cheese.
Spoon the filling back into the potatoes, sprinkle with the remaining cheese and bake for another 15 minutes, until the cheese has fully melted.