Preheat the oven to 350 F. Remove the tops of the Oreo cookies, and set aside. Place the bottoms on a baking sheet.
Top with one mini Reese's peanut butter cup and one marshmallow. Bake for 2-3 minutes. Remove from the oven, and place the tops of the cookies back on.