Lay four slices of pumpernickel bread onto countertop. Spread all slices with spicy brown mustard. Spread horseradish onto one side of each sandwich followed by mayonnaise.
Place two pieces of Swiss cheese onto bottom piece of pumpernickel bread then 2 pieces of corned beef and repeat with another layer of two pieces of Swiss cheese and 2 pieces of corned beef then place the top of the sandwich onto the meat and cheese layers.
Place sandwiches onto cookie sheet pan. Brush each side of sandwich with olive oil. Bake for 15 minutes in 350 degree oven until each side is toasted and cheese is melted.
Remove from cookie sheet pan and place on plate. Serve.