From dumb(“unable to speak”) +waiter, originally separate words and describing the portable table's inability to relate gossip after the meal. By the use of the term to describe small service elevators in American homes in the 1840s, it simply meant dumb as “mechanical”, “unable to speak at all”.
1951, Isaac Jefferson, chapter 9, in Memoirs Of A Monticello Slave, University Of Virginia Press, page 27:
When writing he had a copyin machine: while he was a-writin he wouldn't suffer nobody to come in his room: had a dumb-waiter: when he wanted anything he had nothin to do but turn a crank and the dumb-waiter would bring him water or fruit on a plate or anything he wanted.
Sauces, wines, all the accessories we wanted, and all of the best, were given out by our host from his dumbwaiter; and when they had made the circuit of the table, he always put them back again.