Preheat oven to 350 F. Line a 12 cup muffin pan with cupcake liners. In a large bowl, whisk together flour, sugar, cocoa powder, salt, and baking powder.
Add in eggs, milk, vegetable oil, and vanilla extract, whisk until smooth. Evenly divide cupcake batter between the liners. Bake for 18-22 minutes. Let cool completely.
Place caramel bits and heavy cream in a large microwave safe bowl. Heat in the microwave in 30 second intervals, stirring after each until fully melted and smooth. In a large bowl, with a mixer beat softened butter with powdered sugar, cocoa powder, milk, and vanilla extract, until smooth.
Cut a circle at the top of each cupcake, remove the filling, add in the caramel sauce. Place the top of the cupcake back on top. Pipe frosting onto cupcakes.
