Spivacke was born and grew up in New York City. He studied economics and philosophy at New York University, receiving a BA in 1923 and an MA the following year, and at the same time studied piano privately. Finding the business world uncongenial, he decided to turn to music as a profession. He soon found performing and coaching were not enough and, wishing to explore the philosophy and history of music more deeply, he enrolled at the University of Berlin in 1929, where he studied with Karl Erich Schumann, Arnold Schering, and Curt Sachs. At the same time, he studied piano privately with Eugen d'Albert and composition with Hugo Leichtentritt. After receiving a PhD in musicology in 1933, with a dissertation on the objective and subjective aspects of tonal intensity, he returned to New York.
After working for a short time as a research assistant to Olin Downes at The New York Times, as well as in the Music division of the New York Public Library, late in 1934 he was appointed Assistant Chief of the Music Division at the Library of Congress in Washington, D. C. In 1937 he was promoted to Chief of the division, and remained in that post until he retired in 1972.
As Chief of the Music Division, Spivacke was active with a number of other governmental agencies and departments, as well as with professional organizations such as the Music Library Association (of which he was president from 1951 to 1953), the National Music Council, the International Association of Music Libraries, and the American Musicological Society.
In collaboration with the Elizabeth Sprague Coolidge Foundation, the Gertrud Clark Whittall Foundation, and the Koussevitsky Foundation, the Music Division under Spivacke's leadership sponsored hundreds of concerts of chamber music and commissioned new works from composers such as Aaron Copland, Alberto Ginastera, Roy Harris, Paul Hindemith, Gian Carlo Menotti, Walter Piston, and William Schuman.
He was married twice, first in 1927 to the violinist Carolyn Le Fevre. They divorced in 1953, and two years later Spivacke married the conductor and University of Maryland professor Rose Marie Grentzer.