Biological evolution: maximization of reproduction

Pre-biological types of broadly-defined reproduction are:
Restoration: ability of fermions, atoms, & molecules to maintain their structure while exchanging energy & constituent particles. That’s a spatially- or temporally- discontinuous pattern preservation: replication over time, but not over space (which would be reproduction in conventional sense).
Replication: atoms/molecules are selected by template from an environment, as in restoration. But the new layer then selects the next layer of constituents from the environment, as in crystals, or is detached & then each layer repeats the cycle, as with prions, RNA & perhaps some other nucleic acids.
The selection on these two stages is by direct replicational fitness, with no differentiated phenotype.

Moving up to biology, a good start is "Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life” by Eva Jablonka & Marion Lamb. The book reviews types of “heritable” traits of increasing complexity & variability, & their transmission mechanisms.
I'd quibble that these are stages rather than dimensions: they did not appear all at once but evolved sequentially. More importantly, accelerating variation on higher stages requires new selection criteria for the traits, which E4D addresses only tangentially. Selection by reproductive fitness works for genetic variation, - mutation rates are comparable to the duration of reproductive cycle. But starting with epigenetics, & increasingly so for behavioral & symbolic traits, variation is much faster than reproduction. These traits must be selected by increasingly abstract representational criteria, such as memory of usage, proximity/ similarity to such memory, & predictions thereof by projecting patterns:

Epigenetic modulation: reinforcement of frequently used, & inhibition of the unused functions. Partly heritable, epigenetic memory of usage modulates gene expression in ontogeny of somatic phenotype.
Neural conditioning of　stimuli recognized as reliably associated with usage via some sort of proximity. Conditioning drives the development of more complex & dynamic behavioral patterns, enabling an increasingly differentiated food chain, - an ecological phenotype.
Cortical cognition: discovery & projection of patterns (mutually corresponding stimuli), potentially predictive of usage or its proximity. Such patterns are encoded into predictive models, accumulated & refined across generations via language & recorded media. Symbolic models enable flexible functional differentiation & technological phenotype of human society. To elaborate on each of the above:

Memory of usage: Life forms evolve a differentiated somatic phenotype, which selectively accumulates material & energy to facilitate preservation & reproduction of a genotype. Phenotypic functions are selectively reinforced/inhibited by Lamarckian use & disuse within a reproductive cycle via epigenetic variation. Moreover, such variation is often heritable, shifting initial activity levels across generations (Lamarck has been partly vindicated, see E4D). E4D lists four types of epigenetic inheritance: self-sustaining feedback loops, structural inheritance, chromatic marking, & RNA-mediated inheritance, & there're probably many others. Given that complexity of genotype in phylogeny is growing very slowly, the much faster increase in complexity of somatic phenotype must be driven largely by epigenetics:

Basic life adds an alternative gene expression/transcription layer to a replication cycle, with the genes also serving as templates for RNA, proteins & so on. Gene expression changes in sequence controlled by master genes, modulated by environmental impact via methylation & other types of "imprinting". Such "memory" of usage alters sensitivity to future stimuli by changing the number & responsiveness of corresponding "receptors". When heritable, this memory speeds up ontogenetic development.
Eukaryotes add a more adaptive "cell body" phenotype layer to a relatively conserved nucleus, further sub-differentiating reproductive cycle. The cell body is less "conserved", - it has much faster & more adaptive replacement cycle than the nucleus. In fact, the cell body & some organelles (mitochondria & chloroplasts), were probably acquired by the nucleus (former archaea) from foreign bacteria.
Multicellular organisms add differentiated somatic cells, & corresponding alternative gene expression cycle, to the original germ cells. Somatic phenotype can be hugely more complex & adaptive through the reproductive cycle, & some of this adaptation is also epigenetically heritable.
Sexual dimorphism seems to add a "test" phenotype. Reproductive phenotype is largely female, while males of many species only contribute their genes, serving as "one half of a germ cell". On the other hand, that half is less conserved than the female half: human Y chromosome differs from that of a chimpanzee by ~30%, as opposed to ~2% for the rest of genome. Apparently, males function as a still more adaptive layer of phenotype: a test vehicle for externalized & then sexually selected variation.

Conditioning: Sufficiently advanced somatic phenotype gains controlled spatial mobility, initially guided by inherited & usage-modulated reactions. Mobility leads to the development of higher levels of ecosystem: animals feeding on plants & carnivores feeding on herbivores. In effect, this "food chain" forms an extended phenotype, with "variation" vastly accelerated due to mobility. A correspondingly adaptive new layer of internal phenotype evolves on the higher levels of the food chain: sensors & nervous system. This representational phenotype selects patterns of stimuli to guide behavior by reacting to ecological variation, utilizing initially chemical & then also electrical signaling. However, in a dynamic environment, modulation by past usage alone soon becomes dysfunctional. To be useful, the responses must be modulated by increasingly predictive stimuli. A stimulus is predictive if it consistently co/counter-occurred with inherited, usage-modulated, or previously conditioned stimuli. The former are conditioned to activate or suppress reactions associated with the latter in the past.

Conditioning evolves to potentiate stimuli by increasingly distant & indirect coincidence| precedence:
Direct temporal associations are likely conditioned by amygdale, initially to invoke 4F-type drives.
More complex spatial associations of memories seem to be formed by hippocampus, which contains a map of "place cells" for spatial orientation. Another function of hippocampus is forming declarative memories, probably by facilitating their transfer from primary into association cortices. So, my interpretation is that hippocampus "conditions" memories by associating them with value-loaded locations (in evolutionary context “valuable” must be, or likely to become, close to the subject). Without such association memories are not recognized as important, thus do not become declarative. Discovery & conditioning of still more abstract associations probably requires neocortex. These include operant conditioning of actions by their consequences, & instrumental conditioning of “tools" by their products. Among higher social animals conditioning is "heritable" by imitation, which accelerates acquisition of behavior & facilitates development of "animal traditions".

Cognition: Predictive modeling adds a new layer to representational phenotype: patterns of value-free stimuli. The selection criterion for these patterns is correspondence, quantifiable as cumulative match among inputs, which indicates their predictive power. Such patterns may later be conditioned if their projection is associated with correspondingly projected value patterns. This probably requires some form of cortical minicolumns. Cognitive phenotype beyond neocortex may include oral & recorded language, as well as institutional culture & science: Jablonka's "symbolic dimension". But a more effective selection criterion, - correspondence vs coincidence, is far more important because it outlast any specific transmission method for its selections. Conditioning can also be transmitted via external media, but by itself it would never produce anything like human civilization. Cognitive prediction & planning enables vastly expanded external phenotype: ecological, by cultivating plants & animals, social, via dynamic functional differentiation, & technological, by manufacturing structures & tools.

I won't get into detail on ecological & technological phenotype hierarchies (as I did with somatic hierarchy). That would be too much of a distraction, & I am sure anyone smart enough to read this can conjure his own versions. I’ll also skip “societal phenotype” – it’s too complex for a brief treatment. Again, these are layers of operational phenotype, acquired by a far more fundamental representational phenotype: mechanisms of conditioning & pattern recognition.
A similar interpretation of ecosystem & technosphere is explored in Richard Dawkins' excellent "The Extended Phenotype: The Long Reach Of The Gene". But beside the genes, I think such "phenotype" also propagates more abstract & functional conserved cores: mechanisms of response reinforcement or inhibition by usage, coincidence, & prediction. Such mechanisms should displace genes as the ultimately conserved core as soon as a system develops more efficient reproduction media.

To reiterate: The genome becomes more stable in higher organisms: mutation rates are the highest in bacteria & decrease as the organisms become larger & longer-living. At the opposite extreme, humans are exceptionally uniform genetically. This is because of improved protection for genome, as well as longer reproductive cycle (mutations accumulate during reproduction). Despite this slowdown of genetic variation, the growth of phenotypic complexity appears to accelerate during phylogeny. This can only be explained by increasing proportion of complexity selectively acquired during ontogeny, according to increasingly mediated representational fitness. 

Reproductive fitness is still the ultimate arbiter in biology, but the shift from innate to conditioned motives begins a transition to correspondence-maximizing phase of metaevolution. This transition would be complete with the next shift: from conditioned motives to purely cognitive curiosity, discussed in my next post: