I see a WWI Draft Registration card (attached) which gives an address of Cardiff, Alabama in 1918.
I looked at ED 126 for the 1920 which covers Cardiff, but did not see him there.
You might check for a birth certificate for Thomas (1919) and compare it to the address of the mother shown on Mattie's 1921 birth certificate. Then go line-by-line on the 1920 for that location, or both locations if they do not match.
Unfortunately, sometimes people were just missed, but that is what I would do to confirm it.
You might also check the deeds in Jefferson County to see if they owned any land at that time.
I do note that the draft registration card gives his next of kin as his mother--usually I have seen the spouse given in this instance, so perhaps they were living apart at the time due to work.
Hope this helps,