To make the filling, place all dry ingredients in a saucepan and whisk to combine. Then add pumpkin puree and whisk again. Slowly pour in milk and stir again until well combined.
Place over medium heat and bring to a low bubble - not boil - whisking often. Once it starts bubbling and getting thick, reduce heat to medium-low and continue cooking until a visible ribbon forms when spooning it across the top. It should be thick and kind of jiggly. During this process switch to a rubber spatula for stirring to ensure the pudding isn't sticking to the bottom or sides of the pan.
Remove from heat and add vanilla and whisk. Let set for 5-10 minutes, then transfer to a glass bowl and cover with plastic wrap, making sure the wrap TOUCHES the pudding - otherwise a film will form.
Refrigerate for several hours or until completely chilled and set.