In a small bowl, combine cracker crumbs and sugar; stir in butter. Set aside 1/2 cup for topping. Press remaining crumb mixture into bottom and up the sides of a greased 9 inch springform pan or 9 inch square baking pan. Bake for 5-7 minutes. Cool on wire rack.
In a large bowl, beat cream cheese and sugar until smooth. Fold in 2 cups whipped topping. Arrange half of the banana slices in crust; ttop with half of the cream cheese mixture. Repeat layers.
In a small bowl, whisk milk and pudding mix for 2 minutes. Let stand for 2 minutes or until soft-set; fold in remaining whipped topping. Pour over the cream cheese layer. Sprinkle with reserved crumb mixture. Refrigerate for 1-2 hours or until set.