Real-time social selection maintains honesty of a dynamic visual signal in cooperative fish

<jats:title>Abstract</jats:title><jats:p>Our understanding of animal communication has been largely driven by advances in theory since empirical evidence has been difficult to obtain. Costly signaling theory became the dominant paradigm explaining the evolution of honest signals, according to which communication reliability relies on differential costs imposed on signalers to distinguish animals of different quality. On the other hand, mathematical models disagree on the source of costs at the communication equilibrium. Here, we present an empirical framework to study the evolution of honest signals that generates predictions on the form, function, and sources of reliability of visual signals. We test these predictions on the facial color patterns of the cooperatively breeding Princess of Burundi cichlid, Neolamprologus brichardi. Using theoretical visual models and behavioral experiments we show that these patterns possess stable chromatic properties for efficient transmission in the aquatic environment, while dynamic changes in signal luminance are used by the fish to communicate switches in aggressive intent. By manipulating signal into out-of-equilibrium expression and simulating a cheater invasion, we demonstrate that social costs (receiver retaliation) promote the honesty of this dynamic conventional signal. By directly probing the sender of a signal in real time, social selection is likely to be the mechanism of choice shaping the evolution of inexpensive, yet reliable context-dependent social signals in general.</jats:p>