FairMormon is a non-profit organization dedicated to providing well-documented answers to criticisms of LDS doctrine, belief and practice.
Question: Was Joseph Smith's relationship with Fanny Alger an affair, or was it actually his first plural marriage?
Question: Was Joseph Smith's relationship with Fanny Alger an affair, or was it his first plural marriage?
It seems clear that Joseph, Fanny's family, Levi Hancock, and even hostile witnesses saw their relationship as a marriage, albeit an unorthodox one
Critics charge that Joseph Smith's early plural marriage(s) cannot have been "real" marriages, since the doctrine of "eternal marriage" (i.e., marriages which last beyond the grave) was not introduced until 1841. The Fanny Alger marriage illustrates many of the difficulties which the historian encounters in polygamy. There is little information available, much of it is second hand, and virtually all of it was recorded "after the fact." Even the dates are unclear, and subject to debate.
It seems clear, however, that Joseph, Fanny's family, Levi Hancock, and even hostile witnesses saw their relationship as a marriage, albeit an unorthodox one. The witness of Chauncey Webb and Ann Eliza Webb Young make it untenable to claim that only a later Mormon whitewash turned an affair into a marriage.
Gary J. Bergera, an advocate of the "affair" theory, wrote:
I do not believe that Fanny Alger, whom Compton counts as Smith’s first plural wife, satisfies the criteria to be considered a “wife.” Briefly, the sources for such a “marriage” are all retrospective and presented from a point of view favoring plural marriage, rather than, say, an extramarital liaison…Smith’s doctrine of eternal marriage was not formulated until after 1839–40. 
There are several problems with this analysis. While it is true that sources on Fanny are all retrospective, the same is true of many early plural marriages. Fanny's marriage has more evidence than some. Bergera says that all the sources about Fanny's marriage come "from a point of view favoring plural marriage," but this claim is clearly false.
Hostile accounts which reported a marriage or sealing
For example, Fanny's marriage was mentioned by Ann Eliza Webb Young, a later wife of Brigham Young's who divorced him, published an anti-Mormon book, and spent much of her time giving anti-Mormon, anti-polygamy lectures. Fanny stayed with Ann Eliza's family after leaving Joseph and Emma's house, and both Ann Eliza and her father Chauncey Webb  refer to Joseph's relationship to Fanny as a "sealing."  Eliza also noted that the Alger family "considered it the highest honor to have their daughter adopted into the prophet's family, and her mother has always claimed that she [Fanny] was sealed to Joseph at that time."  This would be a strange attitude to take if their relationship was a mere affair. And, the hostile Webbs had no reason to invent a "sealing" idea if they could have made Fanny into a mere case of adultery.
Plural marriage was not the same as eternal marriage
Bergera is also mistaken in combining the ideas of plural marriage and eternal marriage. It is true that these ideas were later treated as parts of the same doctrine; however, it is fairly clear that Joseph was teaching a few about plural marriage by 1831 (see here). The idea that marriages could last beyond the grave seems to have come later—but, still not as late as Bergera assumes. On May 26, 1835, WW Phelps wrote to his wife of "[a] new idea, Sally, if you and I continue faithful to the end, we are certain to be one in the Lord throughout eternity; this is one of the most glorious consolations we can have in the flesh."  In June, Phelps discussed the doctrine in print, writing:
- We shall by and bye learn that we were with God in another world, before the foundation of the world, and had our agency: that we came into this world and have our agency, in order that we may prepare ourselves for a kingdom of glory; become archangels, even the sons of God where the man is neither without the woman, nor the woman without the man in the Lord: A consummation of glory, and happiness, and perfection so greatly to be wished, that I would not miss of it for the fame of ten worlds. 
He reinforced this "new idea" in a September 1835 letter to his wife, declaring that he would have a "right to" her "in the world to come according to the law of the celestial kingdom.” 
Even if Bergera's estimate was not five years too late, and eternal marriage (i.e., marriage relationships which survive beyond the grave) was not "formulated until after 1839–1840," this would not mean that teachings about plural marriage did not appear until the 1840s. Plural marriage and eternal marriage are distinct concepts, which were formally merged by the revelation(s) included in D&C 132, but they had a long history in Joseph's thought before then.
William Clayton provides an excellent illustration of the conceptual division between the two types of marriage. He entered into plural marriage on April 27, 1843, but would write nearly a month later of his desire "to be united in an everlasting covenant to my [first] wife and pray that it may soon be."  Even for Clayton in 1843, plural marriage and a legal first marriage did not subsume or eliminate eternal marriage with the first partner.
Similarly, Parley P. Pratt married his plural wife Belinda on 20 November 1844. Belinda was not endowed until their return from an eastern mission in August 1845, at which point they were again sealed.  Clearly, in the early Church plural marriage was conceptually separate from eternal marriage, though the doctrines were related. We cannot, then, use the appearance of one to date the introduction of the other.
Some have wondered how the first plural marriages (such as the Alger marriage) could have occurred before the 1836 restoration of the sealing keys in the Kirtland temple
Some have wondered how the first plural marriages (such as the Alger marriage) could have occurred before the 1836 restoration of the sealing keys in the Kirtland temple (see DC 110:). Again, this confusion occurs because we tend to conflate several ideas. They were not all initially wrapped together in one doctrine:
- plural marriage - the idea that one could be married (in mortality) to more than one woman: being taught by 1831.
- eternal marriage - the idea that a man and spouse could be sealed and remain together beyond the grave: being taught by 1835.
- "celestial" marriage - the combination of the above two ideas, in which all marriages—plural and monogamous—could last beyond the grave via the sealing powers: implemented by 1840-41.
Thus, the marriage to Fanny would have occurred under the understanding #1 above. The concept of sealing beyond the grave came later. }}
- Gary James Bergera, "Identifying the Earliest Mormon Polygamists, 1841–44," Dialogue: A Journal of Mormon Thought 38 no. 3 (Fall 2005), 30n75.
- Wilhelm Wyl, [Wilhelm Ritter von Wymetal], Mormon Portraits Volume First: Joseph Smith the Prophet, His Family and Friends (Salt Lake City, Utah: Tribune Printing and Publishing Company, 1886), 57; Ann Eliza Young, Wife No. 19, or the Story of a Life in Bondage, Being a Complete Exposé of Mormonism, and Revealing the Sorrows, Sacrifices and Sufferings of Women in Polygamy (Hartford, Conn.: Custin, Gilman & Company, 1876), 66–67; discussed in Danel W. Bachman, "A Study of the Mormon Practice of Polygamy before the Death of Joseph Smith" (Purdue University, 1975), 140 and Compton, In Sacred Loneliness, 34–35.
- Ann Eliza would have observed none of the Fanny marriage at first hand, since she was not born until 1840. The Webbs’ accounts are perhaps best seen as two versions of the same perspective.
- Young, Wife No. 19, 66–67; discussed by Bachman, "Mormon Practice of Polygamy", 83n102; see also Ann Eliza Webb Young to Mary Bond, 24 April 1876 and 4 May 1876, Myron H. Bond collection, P21, f11, RLDS Archives cited by Compton, In Sacred Loneliness, 34 and commentary in Todd Compton, "A Trajectory of Plurality: An Overview of Joseph Smith's Thirty-Three Plural Wives," Dialogue: A Journal of Mormon Thought 29/2 (Summer 1996): 30.
- W.W. Phelps to Sally Phelps, letter (26 May 1835), off-site
- WW Phelps, "Letter No. 8," Latter Day Saints' Messenger and Advocate 1 no. 9 (June 1835), 130. (italics added)
- WW Phelps to Sally Phelps, letter (16 September 1835); cited in Van Wagoner, Mormon Polygamy, 3 n. 2.
- William Clayton Journal (16 May 1843).
- David J. Whittaker, "Early Mormon Polygamy Defenses," Journal of Mormon History 11 (1984): 54.