Bring a large pot of water to a boil. Cook the potatoes, carrots, and beets in the boiling water until cooked but still firm, 15 to 20 minutes for the potatoes, 25 to 30 minutes for the carrots, and 40 minutes for the beets. You may stagger the start times.
Allow each to cool to the touch before further handling. Remove and discard the skins from the potatoes and beets. Shred the potatoes, carrots, and beets, keeping them separate.
Arrange about half the onion and half the herring fillets in a layer in the bottom of a glass bowl. Cover the layer with about half of the grated potatoes, then a layer of about half the carrots, and finally about half the beets.
Spread about 3 Tbsp. mayonnaise over the salad; season with salt and pepper. Repeat the layers in the same order to finish building the salad. Cover with plastic wrap and chill in the refrigerator at least 1 hour. Serve cold.