It takes a lot of thrust and energy expended just to get the rocket to hover stationary above the pad. The delta-V lost is called the gravity loss. As the rocket gets closer to orbital velocity, gravity losses drop. At low velocities, gravity losses are just time * acceleration by gravity.
Gravity losses, as it turns out, are a big deal. As you add propellant, you logarithmically increase the delta-V, but you linearly increase the gravity losses. At some point, gravity losses overtake the increased delta-V.
The tradeoff between the two is different for different kinds of rockets. In a solid-fuelled rocket, the entire stage is a big combustion chamber, that must contain the gas pressure used to accelerate the vehicle upward. Large pressure vessels are heavy, and so the fuel container is a large fraction of a solid rocket's mass. Pressure-fed liquid-fuelled rockets don't actually have the combustion chamber in the tanks, but the tanks must hold higher pressure than the combustion chamber, so the mass penalty is similar.
Pump-fed liquid-fuelled rockets hold their propellants at a small fraction of the combustion chamber pressure, and so their tanks are a small fraction of the weight of a similarly-sized solid rocket.
Let's take a look at the two extremes. First, a kerosene/LOX liquid-fuelled rocket, like the Saturn V or Falcon 9. I've made up a table to show the decreasing performance return of steadily larger and larger tanks. Here I've presumed a base rocket with an Isp of 290, a first-stage thrust of 750,000 kg, a Gross Lift-Off Weight (GLOW) of 500,000 kg, and a first stage burnout weight (this includes the upper stages) of 150,000 kg. This rocket has an initial acceleration of 1.5 G (but remember you lose 1 G to earth). Incremental tankage weighs just 2.5% of the incremental propellant stored. I'm assuming that the engine thrust stays fixed. Delta-V numbers are in meters/sec. Tower clearance times are a little high, as they assume no acceleration beyond the initial acceleration.
You can see why the Saturn V initial acceleration was just 1.13 Gs. You can also see why launching the Saturn V in a strong wind could have been a problem: it takes a long time to get past the tower.
Now let's look at the other extreme, a solid rocket first stage, like the Titan 4 or the Stick proposal. (Because most of the Shuttle's liftoff thrust is from it's solids, it fits in this category too.) Here the Isp is a bit lower, but more importantly the tankage fraction is far higher: 12%.
You can see why a Titan 4 gets off the pad with nearly 1.5 Gs of initial acceleration.
Solid rockets are a good match for first stage engines. It takes relatively little engineering work to produce a large amount of thrust from a solid motor, which is the biggest cost driver for first stage engines. But because the casing weighs so much, and also because solid propellants have lower Isp, the delta-V of a solid first stage is never going to be as good as a liquid-fuelled analog. That leaves more work for the second stage, which means thinner engineering margins and more desire for high-Isp propellants, like liquid hydrogen. And so, cheap solid rocket first stages drive more cost into the upper stage.