Preheat the oven to 350 F. Remove the tops off of the Oreo cookies and set aside. Place the bottoms on a baking sheet. Top with a mini Reese's peanut butter cup and marshmallow.
Bake for 2-3 minutes. Remove from the oven, and sandwich with the top of each Oreo cookie.