Heat oven to 375°. Combine 3/4 cup sugar and 1 1/2 tbls. water in a 10-inch ovenproof skillet and stir until the mixture looks like wet sand. Cook over medium-high heat, gently swirling the pan occasionally but not stirring, until light amber, about 5 minutes. Remove from the heat and arrange the peaches in concentric circles over the caramel. Drizzle with the bourbon; set the skillet aside.
If you are using the vanilla beans, scrape the seeds into the buttermilk and set aside.
Whisk the flour, ground pecans, baking powder and baking soda and salt in the bowl. Beat the butter and the remaining 1 cup flour in a large bowl with a mixer on med speed until light and fluffy, about 3 minutes. Add the eggs one at a time, beating after each addition, then beat in the vanilla. Reduce the mixer speed to low; add the flour mixture in 3 batches, alternating with the butter milk, beginning and ending with the flour, and mix until just incorporated. Pour the batter over the peaches and spread evenly.
Transfer the skillet to the oven and bake until the cake is golden and a toothpick inserted into the center comes out clean, about 50 minutes. Cover loosely with foil if the cake is browning to quickly. Let sit for 15 minutes then invert onto a platter and let cool completely.