unfortunately, it's the toppings that are high in calories that often make it on your potato. going heavy on sour cream, butter and cheese can double the number of calories and fat, which means, you no longer have a healthy baked potato.