AI knows what it knows because it has "learned" from everything out there in the vast internet. The problem is that a lot of what is out there is wrong. You could probably ask it to prove that the world is flat and it will recall all those ancient texts that said it was. It's always fun when you type a text and it tries to create the word before you type it--some hilarious guesses come up.
I do think AI could create a perfect tax return. It could "learn" the entire US Code Title 26 and all the regulations, court cases, and rulings. As our tax system has gotten more complex, we tax pros have often lamented that there is no way one person could know it all. Now some of us will not prepare returns that have foreign income, crypto transactions, etc. or tackle estates, trusts, and worst of all partnerships, because we do not think we have the knowledge to be accurate. Eventually AI could handle it. Tax pros will join the ranks of milkmen, lamplighters, keypunch and telephone operators in the Museum of Extinct Professions.