Ideally, the size of the box depends on what's in the box. Though I have seen a few absurdly overpriced products with an equally absurd amount of empty space inside, most of the smaller boxes are stripped down in terms of circuitry and features. That can be good or bad, depending on what you want and need.
1. Most important question. Knobs? Some people want to control their mix with their hands on faders. If the box has knobs or faders, I would call it a "mixer" as opposed to a "summing device" (which is a mixer without knobs). If you want to maintain the recall and automation ability of the DAW (which is the whole point of having a computer in your studio at all, IMHO) then you want a summing device that does not have knobs. Score one point for the small boxes. If not, then buy a mixing console.
2. Switches. Passive summing devices in particular depend on seeing a reliably low source impedance on every channel to maintain consistent insertion loss, crosstalk, and noise performance. If you're ever going to use a different number of inputs or assign them differently in the stereo spectrum, then you need some way of configuring each channel to feed the left or right busses (or both), and to terminate unused inputs. The most cost-effective way to do that is with switches. They take up some space, but in my opinion they're mandatory. I score one point for the not-quite-so-small.
3. Active circuitry. The small boxes are the passive ones. If you feel like you need buffer amps between your inputs and the summing node, then you'll have some extra circuitry in the signal path and you'll have a power supply and such. My design choice was to omit what was not needed, for a shorter, cleaner, more transparent signal path. Although this makes the summing box smaller, it also requires external makeup gain, so the total system isn't really any smaller than the active boxes. The big selling point for passive summing is that you get to choose the flavor of your makeup gain, as opposed to being stuck with one sound in the active box (though it may be a good one). My vote obviously is for the passive, but it's a matter of taste.
4. Flavor. Transformers and tubes and discrete op amps can be tossed into any circuit for flavor. The question is, do you want them on every channel all the time, or do you want to patch them in to channels or busses as you see fit? That's a question of preference and workflow, and there's no wrong answer. My personal feeling is that it's a waste of money to have a transformer-coupled tube amp permanently and irrevocably installed on a channel where you'd occasionally prefer the sound without it. Better to have a rack full of outboard gear (or plug-ins) where you can apply the flavors where and when you need them. Which makes the summing box smaller, but the system bigger (and more fun
So you can see how different designers can arrive at different solutions for the same basic concept, and those solutions can have wildly different form factors. The solution I came up with left out pretty much everything except the switches. I don't think there's anything at all missing from the Folcrom, and I don't think there's anything that can be removed without crippling it.