Cook bacon until crispy. Drain, reserving a few tablespoons of the grease, and crumble. Set aside.
Place the potatoes in a large saucepan. Cover with cold water. Add a tablespoon of the bacon grease and some salt and place over medium-high heat. Bring to a boil and cook until the potatoes are tender, about 15 to 20 minutes. Drain and let the potatoes cool and then cut into bite-size pieces.
In a separate bowl, mix mayonnaise, Dijon mustard, salt, and pepper for dressing. Toss potatoes, onion, celery and egg in dressing. Add bacon. Serve chilled.