it actually just refers to the flavor of a vanilla custard (aka the classic french way of making ice cream from cream and egg yolks). french vanilla cake mimics a custard flavor by including egg yolks in the batter, making it a yellow cake.