In a medium bowl, blend powdered sugar, butter, milk and coffee, adjusting powdered sugar and milk as needed to create a smooth frosting. Divide frosting into four smaller bowls. First flavor each bowl, adjusting amount of flavored extract to taste. After desired flavor is achieved, color each bowl with a few drops of food coloring. Traditionally, the colors are kept muted, but they can be made as bright as desired. Once the frosting is colored, frost cooled cookies.