I’ve done both, but yes, mostly I’ve quit because of managers. Second in line is coworkers (and that’s really on the managers as well). A distant last is the one time I quit a job because I didn’t like the job itself.