This perspective, along with a number of taciturn observations made by Kaufman himself, led to a simplistic but commonly held belief that Hart was the emotional soul of the creative team while Kaufman was a misanthropic writer of punchlines. Kaufman preferred never to leave Manhattan. He once said: "I never want to go any place where I can't get back to Broadway and 44th by midnight."<ref>{{cite book| last=Meryman| first=Richard| author-link=Richard Meryman|title=Mank: The Wit, World, and Life of Herman Mankiewicz| url=https://archive.org/details/mankwitworldlife00mery| url-access=registration|publisher=William Morrow|year=1978|location=New York|page=[https://archive.org/details/mankwitworldlife00mery/page/100 100]| isbn=9780688033569}}</ref>
Called "Public Lover Number One", he "dated some of the most beautiful women on Broadway".<ref name=wallace174>Wallace 2008, p. 174.</ref> Kaufman found himself in the center of a scandal in 1936 when, in the midst of a child custody suit, the former husband of actress [[Mary Astor]] threatened to publish one of Astor's diaries purportedly containing extremely explicit details of an affair between Kaufman and the actress.<ref name=wallace174/> The diary was eventually destroyed by the court, unread, in 1952, but details of the supposed contents were published in ''[[Confidential (magazine)|Confidential]]'' magazine, ''[[Hollywood Babylon]]'' by [[Kenneth Anger]] (Bothboth always have been considered unreliable sources)<ref>[https://latimesblogs.latimes.com/thedailymirror/2007/08/confidential--1.html ''Los Angeles Times'' piece about unreliability of ''Confidential'' magazine]</ref><ref>[http://thepurplediaries.com/2017/03/29/kenneth-angers-hollywood-babylon-fiction-verses-fact/ RS explains unreliability of Kenneth Anger’s Hollywood Babylon]</ref> and in various other questionable publications. Some of the sexually explicit portions of Mary Astor’s writing about Kaufman were reprinted in ''[[New York (magazine)|New York]]'' magazine in 2012 and ''[[Vanity Fair (magazine)|Vanity Fair]]'' magazine in 2016.<ref name=NYmag>{{cite journal|title=Mary Astor Blushes When Her Filthy Diary Leaks|journal=[[New York (magazine)|New York]]|date=April 9, 2012|page=44|url=http://nymag.com/news/features/scandals/mary-astor-2012-4|access-date=September 26, 2016}}</ref><ref>{{cite journal|url=http://www.vanityfair.com/hollywood/2016/09/inside-the-trial-of-actress-mary-astor-hollywoods-juiciest-sex-scandal|title=Inside the Trial of Actress Mary Astor, Old Hollywood's Juiciest Sex Scandal|last=Sorel|first=Edward|author-link=Edward Sorel|journal=[[Vanity Fair (magazine)|Vanity Fair]]|date=October 2016 |access-date=September 26, 2016}}</ref> Kaufman had an affair with actress [[Natalie Schafer]] during the 1940s.<ref>{{cite news |last=Brozan |first=Nadine |date=February 13, 1995 |title=Chronicle |url=https://www.nytimes.com/1995/02/13/nyregion/chronicle-058495.html |work=The New York Times |access-date=March 14, 2018 }}</ref>
Kaufman joined the theater club, The Lambs, in 1944.<ref>
