In 1933, Chris and Fannie Baker opened Baker's as a lunchtime sandwich restaurant. In 1934, their son Clarence Baker began booking jazz pianists, but Baker's was still known at that time principally as a restaurant. In 1939 Clarence, took over ownership after Chris had suffered from a stroke. That same year, Clarence began booking pianists from outside the Detroit area, although the club featured local pianist Pat Flowers from 1940 until 1954. In 1952, the club was expanded and remodeled to the Art Deco look that it retains today. By 1954, the business had rapidly expanded, and by the following year, Baker's began featuring major jazz acts, notably Art Tatum who played the last two years of his life, including his last performance in April 1956, Dave Brubeck in 1957 and Gerry Mulligan in 1958. During the 1950s Modern jazz was less common at Baker's than in was in the 1960s, when the emphasis of the club's music changed to Hard Bop. During the 1960s Clarence leased the club out, but resumed personal control in the 1970s.
Baker's is noted for its long history of presenting local and major jazz acts, its excellent acoustics, its intimacy - seating only 99, its Art Deco furnishings, including a distinctive, piano-shaped bar painted with a keyboard motif, Art Deco style paintings of European city landscapes by Harry Julian Carew, tilted mirrors that allow patrons to view the pianist's hands, and its Steinway piano which was selected and purchased in New York by Tatum for Clarence in the 1950s. The club still displays its original liquor pricelist from 1934, showing the price of beer at 26 cents.
In 1986, Baker's was designated as an Historic Site by the Michigan State Historic Preservation Office. The historic significance of the site is stated as follows:
Baker's Keyboard Lounge has significance as Michigan's jazz mecca and Detroit's oldest jazz club in continuous operation. Founded in May 1933 by Chris Baker as a restaurant and piano bar, the present jazz orientation of the club has been firmly in place since 1939. Baker's Keyboard Lounge has hosted the greatest names in blues and jazz since that date. Some of the musicians who have played the club include: Ella Fitzgerald, Miles Davis, Oscar Peterson, George Shearing, Sarah Vaughn, Joe Williams, Maynard Ferguson, Cab Calloway, Woody Herman, Modern Jazz Quartet, and Nat "King" Cole; to name but a few.
In 1984, Baker's Keyboard Lounge celebrated fifty-years of the sound of jazz in Detroit and Michigan.