If White privilege is defined this way, I do agree that Whites, including poor Whites, do have White privilege. The only thing I have against the term is that it may be used as if Whites are the only group that have privilege, or that this is the only kind of privilege people should be more aware of. For people of color, there is indeed White privilege, but for Muslims and Jews, there is also “Christian privilege,” in that Christians do not need to be afraid of certain things such as anti-Muslim attacks or anti-Semitic incidents that Muslims and Jews need to be concerned about. There is also “Straight privilege,” in that discrimination in such things as employment and housing is not something that straight people need to fear, but something that gay people are still aware of and concerned about.
Whites, Christians, and straight people do of course have worries and concerns, financial problems, health concerns, and whatnot; but they do not have the additional fears and worries that come with being people of color, non-Christians, or gay.