Position a rack in the center of the oven and preheat to 350°F (175°C). Line two baking sheets with parchment paper or silicone mats and set aside.
In a medium bowl, whisk together the flour, baking powder, salt, nutmeg, and cinnamon until the spices are evenly spread through the flour. Set aside.
In a stand mixer or large bowl with a hand mixer, beat the softened butter and granulated sugar on medium speed for 2 to 3 minutes until pale and fluffy. Scrape down the sides once.
Add the egg yolks and beat until fully combined. Mix in the vanilla extract and rum extract if using, then pour in the eggnog and mix on low until blended. Scrape the bowl again.
With the mixer on low, add the dry ingredients and mix just until no dry streaks remain and a soft dough forms. Do not overmix.
Use a medium cookie scoop or heaping tablespoon to portion mounds of dough onto the lined sheets, spacing them about 2 inches apart. Press each mound down gently with the back of a spoon for a flatter cookie.
Bake one sheet at a time for 10 to 12 minutes, until the edges look set and the tops are no longer wet. The centers should still feel soft and should not brown. Rotate the pan halfway through if needed.
Let the cookies rest on the hot pan for 5 minutes, then transfer to a wire rack and cool completely before glazing.
In a small bowl, whisk the sifted powdered sugar with 3 tablespoons eggnog and ¼ teaspoon nutmeg until smooth. Add more powdered sugar for a thicker glaze or the last tablespoon of eggnog for a thinner drizzle.
Spoon or drizzle the glaze over the fully cooled cookies, or dip the tops directly into the bowl. Sprinkle with a little extra nutmeg while the glaze is still wet. Let stand for about 15 minutes until the glaze sets into a dry finish.