Preheat the oven to 350°F (175°C). Grease a 9x5 inch loaf pan with butter or non-stick spray and line with parchment paper, leaving an overhang on the long sides.
Peel the peaches using a swivel peeler or paring knife. Cut each peach in half, remove the pit, and dice the flesh into 1/2 inch pieces. Toss the diced peaches in the reserved 1 tablespoon of flour until lightly coated, then set aside.
In a medium bowl, whisk together the 2 cups of flour, baking powder, baking soda, salt, and cinnamon. Set aside.
In a large bowl, beat the softened butter and granulated sugar together with a hand mixer on medium speed for 2 to 3 minutes, until pale and fluffy. Scrape down the sides of the bowl partway through.
Add the eggs one at a time, beating well after each addition. Mix in the vanilla extract and sour cream until fully combined and smooth.
Add the dry flour mixture to the wet ingredients in two batches, stirring by hand with a spatula until just combined. Stop mixing as soon as no dry streaks of flour remain.
Gently fold the flour-coated peaches into the batter using a spatula, turning the bowl as you go rather than stirring vigorously.
In a small bowl, whisk together the sugar, flour, cinnamon, and nutmeg for the topping. Add the cold, cubed butter and cut it in with a pastry cutter or fork until the mixture looks like coarse, pea-sized crumbs.
Pour the batter into the prepared loaf pan and smooth the top with a spatula. Sprinkle the crumb topping evenly over the batter, pressing it in very lightly so it adheres. Bake for 55 to 65 minutes, tenting the top loosely with foil after 40 minutes if the crumb topping is browning too quickly. The bread is done when a toothpick inserted in the center comes out with a few moist crumbs but no wet batter.
Let the loaf cool in the pan for 15 minutes, then use the parchment overhang to lift it onto a wire cooling rack. Cool for at least 30 minutes before slicing. If using the glaze, whisk the powdered sugar, milk, and vanilla together, then drizzle over the cooled loaf.