Preheat oven to 400°F. In a medium saucepan, bring water, butter, and salt to a boil over medium-high heat. Add flour all at once and stir quickly with a wooden spoon until mixture forms a ball; remove from heat.
Add 1 egg to mixture and beat hard with wooden spoon to blend. Add remaining eggs one at a time, beating well after each addition; each egg must be completely blended before the next egg is added. While beating dough, its consistency will change from looking almost curdled to smooth.
When dough is smooth, spoon it into a large resealable plastic storage bag and, using scissors, snip off one corner of bag, making a 1-inch-wide cut. Gently squeeze bag to form ten 1" x 4" dough logs about 2 inches apart on an ungreased large rimmed baking sheet. Bake 40 to 45 minutes, until golden and puffy. Remove to a wire rack to cool.
In a large bowl, beat cream until stiff peaks form. Fold in pudding mix and set aside. In a small bowl, stir together confectioners' sugar, cocoa, and milk to make a smooth icing.
Cut eclair shells horizontally in half and spoon pudding mixture equally into bottom of each. Replace tops of eclair shells and spread cocoa icing over tops. Cover loosely and chill at least 1 hour before serving.