Preheat your oven to 375°F (190°C). Line two baking sheets with parchment paper.
In a large mixing bowl, cream together the softened butter, brown sugar, and granulated sugar until light and fluffy.
Add the egg, egg yolk, and vanilla extract. Beat until fully combined.
Mix in the baking powder, baking soda, salt, cinnamon, and cornstarch.
Gradually add the flour and mix until a soft dough forms. Avoid overmixing.
Scoop the dough into large portions, about 3 tablespoons each, and place them on the prepared baking sheets.
Use your thumb or the back of a spoon to create a shallow indentation in the center of each dough ball.
In a separate bowl, combine the flour, brown sugar, cinnamon, salt, and softened butter for the crumb topping. Mix until crumbly.
Fill each indentation generously with the crumb mixture.
Bake the coffee cake cookies for 9–12 minutes, or until the edges turn lightly golden.
Allow the cookies to cool on the baking sheet for 5 minutes before transferring to a wire rack.
For the glaze, whisk together powdered sugar, cinnamon, and milk until smooth. Adjust the consistency as needed.
Once the cookies are completely cool, drizzle the glaze over the tops and allow it to set before serving.