Cowdery objected to a lot. In the beginning, it was with Joseph violating the separation of Church and state. Later, it was also over Fanny Alger
“Finally, in January 1838, Cowdery wrote his brother Warren that he and Joseph Smith had “had some conversation in which in every instance I did not fail to affirm that which I had said was strictly true. A dirty, nasty, filthy affair of his and Fanny Alger’s was talked over in which I strictly declared that I had never deserted from the truth in the matter, and as I supposed was admitted by himself.” Alger, a teenage maid living with the Smiths, may have been Joseph Smith’s first plural wife, a practice that Cowdery opposed.” Johnson, Benjamin (1903)
Later, his life was threatened: On June 17, 1838, President Sidney Rigdon announced to a large Mormon congregation that the dissenters were “as salt that had lost its savor” and that it was the duty of the faithful to cast the dissenters out “to be trodden beneath the feet of men.” Richard Lyman Bushman, Joseph Smith: Rough Stone Rolling (New York: Alfred A. Knopf, 2005), 349-53
Oliver also wrote a history of the Church that stated that Joseph;s fist encounter with any being was with the Angel Moroni, NOT in a First Vision. This is not surprising since there are so many different versions of the fist vision.