This study investigated the in vitro digestion of purified pea fractions (protein isolate and starch) in sponge cakes when compared to unrefined pea flour and to the whole wheat flour and purified maize starch commonly used in the food industry. Proteins in the wheat cake were hydrolysed more rapidly than those in cakes made with either pea flour or a combination of pea proteins and purified starch. In absolute terms, however, more readily bioaccessible protein was released from these pea cakes (by around 40%). By contrast, cakes containing wheat flour or maize starch were more susceptible to amylolysis compared to those based on pea starch in the form of the purified ingredient or whole flour. This could be attributed to a higher proportion of amylose and resistant starch in the pea cakes as well as structural characteristics that might have decelerated enzyme–substrate interactions. Interestingly, similar digestion patterns were observed regarding the purified pea ingredients and unrefined whole pea flour. It was therefore concluded that pea ingredients, and particularly the less purified and thus more sustainable whole pea flour, are promising plant-based alternatives for use in gluten-free baked products.