Preheat oven to 350 degrees. Place cupcake liners in your cupcake or muffin pan. In a large mixing bowl with a hand mixer or your stand mixer bowl, using the paddle attachment, mix together all of your ingredients except the peanut butter cups for a minute or two until well blended.
Place 3 tbsp of cake batter in each cupcake liner. Take one frozen peanut butter cup and place in the center of each cupcake on top of the batter, press down a little bit.
Bake for 18-22 minutes or until several cupcakes pricked with a toothpick come out clean. Do not prick the cupcake where the peanut butter cup is inserted. There will be a slight indentation or hole where the cupcake is sunken in from the peanut butter cup, you'll fill that with icing. Allow the cupcakes to cool in the pan for 5 to 10 minutes.
Remove cupcakes from the pans and place them on a cooling rack and let them cool all the way. While the cupcakes are cooling, make the buttercream icing.
Icing: In your stand mixer bowl using the paddle attachment, mix thoroughly the softened butter and peanut butter on low to medium speed. Turn off your mixer and in two batches, add the powdered sugar and mix until well blended. Be careful with the powdered sugar - it can go everywhere! Use a pastry bag to pipe the icing onto the cupcakes.