Prepare the flour coating: Pour milk or buttermilk into a shallow bowl. In a second, shallow bowl, mix together flour, salt, and pepper.
Dip the fish into the milk and hold it above the bowl for several seconds to drain. Coat with the seasoned flour, shaking off the excess, and place on a plate. Repeat with remaining fish.
Add 3/4 to 1 inch of vegetable oil to a heavy skilled with high sides or a Dutch oven. Heat to 370 degrees F.
Fry the fish, without crowding, until golden brown on both sides, 5 to 8 minutes, turning once. Drain on paper towels or newspaper.
Place all ingredients in separate bowls. Allow guests to assemble their own tacos.