Place the potatoes in a large pot and cover with water. Bring to a boil and cook until tender but still firm, about 15 minutes. Drain and set in the refrigerator to cool.
Place eggs in a saucepan and cover with cold water. Bring water to a boil and immediately remove from heat. Cover and let eggs stand in hot water for 10 to 12 minutes. Remove from hot water, cool, peel and chop.
Add potatoes to a large bowl, along with the eggs, bacon, onions, and celery. Add mayonnaise, mustard and salt and pepper to taste. Chill for an hour before serving.