Beat butter at medium speed with an electric mixer until creamy; gradually add sugar and brown sugar, beating well. Add flour, cinnamon, and nutmeg; beat just until blended.
2Preheat oven to 350 F. Prepare Streusel Topping.
3Beat butter at medium speed with an electric mixer until creamy; gradually add sugar, beating well. Add eggs, one at a time, beating until blended after each addition.
4Combine flour, baking powder, and salt; add to butter mixture alternately with milk, beginning and ending with flour mixture. Beat at slow speed until blended after each addition. Stir in vanilla. Pour batter in to a greased and floured 9-inch springform pan; top with sliced peaches and blackberries.
5Pinch off 1-inch pieces of Streusel Topping, and drop over fruit.
6Bake at 350 F for 1 hour and 10 minutes to 1 hour and 20 minutes or until center of cake is set. Let cool completely on a wire rack ( about 1 1/2 hours).
7Dust with powdered sugar and garnish, if desired.