what is the difference between a casserole and a souffle? the main difference between a souffle and a casserole is that souffles tend to have a lighter texture and are leavened by egg whites. for this particular dish, the difference between a sweet potato casserole and a souffle is in the texture of the potatoes.