Preheat oven to 350 F. Line a 12 cup muffin pan with cupcake liners. In a large bowl, whisk together flour, sugar, salt, and baking powder. Add in milk, vegetable oil, eggs, and vanilla extract, whisk until smooth.
Evenly divide cupcake batter between the prepared cupcake liners. Bake for 15-18 minutes. Let cool completely.
Beat softened butter with powdered sugar, milk, and vanilla extract until smooth.
Cut the middle out of each cupcake, and place jelly beans into the center of each cupcake, place the top back onto the cupcakes. Pipe frosting onto cupcakes and decorate.