Prepare for baking: Preheat oven to 350°F (175°C). Grease two 9-inch round cake pans and line bottoms with parchment paper. Lightly flour the pans, tapping out excess.
Mix dry ingredients: In a medium bowl, whisk together flour, baking soda, salt, cinnamon, and nutmeg. Set aside.
Cream butter and sugars: In a large bowl, use an electric mixer to cream the softened butter with granulated sugar and brown sugar until light and fluffy, about 3-4 minutes.
Add eggs and vanilla: Beat in eggs one at a time, then add vanilla extract. Mix until well combined.
Incorporate bananas: Add mashed bananas to the butter mixture and mix until just combined.
Alternate wet and dry ingredients: Add the flour mixture in three additions, alternating with buttermilk and sour cream in two additions. Begin and end with flour mixture. Mix just until combined after each addition.
Bake the cakes: Divide batter evenly between prepared pans. Bake for 25-30 minutes, or until a toothpick inserted in center comes out clean or with just a few moist crumbs.
Cool completely: Cool cakes in pans for 10 minutes, then turn out onto wire racks to cool completely before frosting.
Make cream cheese frosting: Beat softened cream cheese and butter until smooth and fluffy. Gradually add powdered sugar, beating until smooth. Add vanilla, salt, and 2-3 tablespoons of cream until desired consistency is reached.
Assemble the cake: Place one cake layer on serving plate. Spread frosting evenly on top. Add second layer and frost top and sides of entire cake.