Cut day old challah bread into thick slices. I got about 8 slices (including ends) from 1 loaf.
In a large dish, whisk together buttermilk, egg yolks, maple syrup, spices, 1 tbsp sugar, and salt.
Add butter and oil to a cast iron pan and allow to slowly heat up on medium-low heat. Dip challah slices into custard and soak for 10-15 seconds per side. Working in batches, gently transfer slices to hot cast iron pan (I fit about 3 slides in my pan at a time). NOTE: the oil is optional, I find it helps the french toast cook more evenly. You can just use butter if you prefer. Pan fry slices for 3 minutes, or until first side is golden brown. Use a spatula to flip slices over, then sprinkle the cooked side of the french toast slices with a generous amount of coarse brown sugar. Cook on the second side for 3 minutes, then flip slices over so the side with the sugar is now face down in the pan. Cook for another 2 minutes, until sugar caramalizes and forms a crust. Repeat with the second side.
Once cooked, transfer the french toast to a plate and let cool for 1-2 minutes, to allow the crust to firm up. Serve with fresh fruits and maple syrup, if desired. You honestly don't even need any maple syrup with this French toast!