Yes, larger trees take longer to establish than smaller ones. Many people assume that the larger the tree, the better. While larger, more expensive trees create an instant landscape, they are slower to establish and grow.
There is a nursery axiom stating that a transplanted tree will take X number of years to re-establish the portion of the root system lost in the digging process. In that axiom, X equals the tree's trunk caliper. For example, a 2-inch-caliper tree will re-establish its root system and be "off and running" in two years; a 4-inch-caliper tree will take four years to recuperate.
Only 5 to 15 percent of a tree's roots are retained when it is dug up and burlapped; 85 to 95 percent of the total root system is left behind. You should not expect a 4-inch-caliper tree to do much growing in the first few years. During that time, it allocates all of its energy to root development. In most cases it is better to spend less money for a smaller tree that will quickly establish and grow to the size of the bigger, more expensive tree.