Preheat oven to 350 degrees. Line cupcake tins with paper liners.
CAKE BATTER: Cream together butter and sugar. Beat in eggs; stir in vanilla. Add flour and baking powder until well mixed. Stir in milk until smooth. Fill cupcake liners 2/3 full with batter. Press one peanut butter cup into each cupcake. Bake for 20-25 minutes. Cool completely before frosting.
FROSTING: Cream butter and peanut butter. Add sugar until combined. Stir in milk as needed until smooth. Frost cooled cupcakes and decorate as desired.