In a small bowl, combine the pepper jelly and mustard; set aside. Season the chicken breasts on both sides with salt and pepper. In a large skillet, melt the butter over moderately high heat. When the foam subsides, add the chicken breasts and cook, turning once, until browned, about 3 minutes per side.
Tilt the skillet and pour off most of the fat. Add 1 tablespoon of water and shake the pan to loosen the browned bits on the bottom. Push the chicken to one side, add the celery matchsticks and cook, stirring, for 1 minute. Add the pepper jelly mixture and the lemon juice to the pan and shake to coat the chicken with the sauce. Cook until the sauce is reduced to a glaze, about 30 seconds. Season with salt and pepper.
Transfer the chicken to plates and spoon the celery matchsticks on top. Garnish with the celery leaves and serve at once.