Place the bacon in a large, deep skillet, and cook over medium-high heat, turning occasionally, until evenly browned, about 10 minutes. Drain the bacon slices on a paper towel-lined plate. Crumble the bacon slices and set aside.
Preheat an oven to 350°F. Spray a baking dish with cooking spray.
Heat the olive oil in a skillet over medium heat. Stir in the chopped mushroom stems, garlic, and jalapeno; cook and stir until the mushrooms release moisture and soften, about 10 minutes. Transfer the mushroom mixture to a bowl, and stir in the cream cheese, cheddar cheese, and bacon. Season with salt and pepper. Spoon the cheese mixture generously into the reserved mushroom caps, and arrange the stuffed caps on the prepared baking dish.
Bake in the preheated oven until cheese begins to brown, 15 to 20 minutes.