In a large bowl, combine first four ingredients; set aside 1/2 cup for topping. Press remaining crumb mixture into an ungreased 13 x 9 inch dish. Chill for 5 minutes.
In a large bowl, whisk milk and pudding mixes for 2 minutes. Let stand for 2 minutes or until set (mixture will be thick). Stir in frozen yogurt and 1 cup whipped topping until smooth. Spread over crust.
Top with remaining whipped topping. Sprinkle with reserved crumb mixture. Refrigerate 8 hours or overnight.