In an 8 or 9 inch cake pan, add 3 tablespoons butter and brown sugar and place in oven while it preheats to 350F. When butter is melted, remove pan from oven. Stir to combine. Spread mixture evenly over bottom of pan.
Arrange apple slices (tightly overlapping) in circle over brown sugar / butter mixture in the pan, using smaller pieces for center. Set aside. In a large mixing bowl, beat eggs at high speed until thick and lemon colored, about 3 minutes. Add white sugar and vanilla, beat another 2 minutes.
While eggs are beating, heat milk and remaining 2 Tbs butter together in a saucepan or microwave, until butter is melted.
In a medium bowl stir together flour, baking powder and salt.
Add flour mixture to beaten eggs and beat just until blended.
Then add hot milk and butter mixture and again beat just until blended.
Pour mixture over apples in pan.
Bake in the preheated oven until the top springs back when lightly touched, about 25 to 35 minutes. Let cool in pan 10 minutes. Loosen edges with sharp knife, then invert and lift off cake pan so cake comes out upside down. Using a spatula or knife, move any apples or sugar mixture left in pan to the top of the cake. Serve warm or at room temperature. Top with whipped cream if you wish.