Posted by: Jeremy Fox | May 13, 2012

An important but little known fact about compensatory dynamics

Many ecologists expect competing species to exhibit compensatory dynamics, meaning that the densities of any two competing species should be negatively correlated over time or across space. If your competitor increases in abundance, you ought to decline, right? After all, to the extent that two species are competing, that means that when one increases, it’s at the expense of the other, right?

Um, no. Or rather, not necessarily. For instance, environmental fluctuations can cause competing species to exhibit positive rather than negative correlations in abundance. Think of a drought which causes the density of every plant species to decline, even though they’re all competing. But there’s a deeper reason why you should not necessarily expect the densities of competing species to all be strongly negatively correlated with one another: in general, it’s mathematically impossible. I don’t think this fact is as well-known as it should be, so I thought I’d post on it.

Say you have just two competitors, each of whose densities you’ve measured at a bunch of different time points, or a bunch of different spatial locations. In this special case, the correlation coefficient (Pearson’s correlation or rank correlation) between the density of species 1 and the density of species 2 can indeed take on any value from +1 to -1. So depending on how strongly the species compete and other factors, it’s possible that their densities could be perfectly compensatory (correlation = -1). So for the sake of illustration, let’s assume that the correlation between their densities is -1.

Now imagine that there’s a third competitor. How will its densities correlate with those of species 1 and 2? Well, to answer that, you’d have to specify more information about the ecology of all three species. But without knowing anything about the ecology, I can tell you what the answer won’t be. Species 3 won’t have a correlation of -1 with both species 1 and 2. Because that’s mathematically impossible. For instance, if species 1 and 3 have a correlation of -1, then by definition species 2 and 3 must have a correlation of +1, i.e. perfectly synchronous rather than perfectly compensatory dynamics. Conversely, if species 3 has correlations of -1 with both species 1 and 2, then by definition species 1 and 2 must have a correlation of +1.

This three species case is a simple illustration of a general principle: the more species you have, the less-compensatory their dynamics can possibly be. It’s mathematically possible for any number of species to all be perfectly in sync with one another. But the more species you have, the less density compensation they can possibly exhibit, on average. In general, we can describe the pairwise correlations among s competitors with a correlation matrix, a square matrix with s rows and s columns, one row and column for each species. The number in row i of column j gives the correlation between species i and j, and of course the same number will appear in row j of column i since the correlation between species i and j is the same as that between j and i. The numbers on the diagonal will all be +1, since by definition any variable is perfectly correlated with itself. Now, as a matter of mathematical necessity, correlation matrices are positive semidefinite. Which turns out to imply that, the larger s is, the less-negative the off-diagonal elements of the correlation matrix can possibly be, on average.

For instance, in the special case when every pair of species has the same correlation, the minimum possible value of that correlation equals -1/(s-1). Here’s the graph for that special case:

As you can see, even with as few as 5 species, in this special case the minimum possible correlation is only -0.25, which is pretty weakly compensatory. In the limit, as s goes to infinity, the minimum possible correlation goes to 0 (i.e. species fluctuate independently of one another).

Of course, in reality the pairwise correlations won’t all be equal, and so even with many competing species it’s possible that some pair of them might have strongly compensatory dynamics. But if they do, that just implies that some other pair of them must have strongly synchronous dynamics. On average, the pairwise correlations can’t be more than slightly negative when you have more than a few species.

Note as well that the same basic point holds for other measures of synchrony. For instance, the exact same points hold if you want to analyze synchrony in the frequency domain by looking at phase differences.

This mathematical fact is certainly familiar to folks who do a lot of work on this stuff, like my collaborator Dave Vasseur. But it deserves to be more widely known. Lots of ecologists have the vague sense that competitors ought to exhibit compensatory dynamics, and so are somewhat surprised to learn that compensatory dynamics are actually quite rare in nature.  But the reason they’re rare is mathematical, not ecological.  Which means you cannot use the rarity of compensatory dynamics as evidence for anything about ecology. For instance, you can’t say “These species only exhibit weakly compensatory dynamics, so they must not be competing very strongly”. You can’t even say “These species only exhibit weakly compensatory dynamics, so environmental fluctuations must be generating synchrony that overrides the strongly compensatory dynamics that would otherwise occur.”

Just to be clear, there absolutely is scope for the strength of synchrony or compensation to vary among communities, and among different pairs of species, for all kinds of interesting ecological reasons. But if you aren’t clear on what dynamics are possible, you’re liable to misinterpret actual dynamics.


  1. Not sure I’m following this one. You switched from the question of whether two species can compensate each other to the question of whether more than two can do so. Those of course are not going to be the same thing, but this is completely trivial. You can still get any two species from some larger group that compensate.

    • That’s the point of the post. Sorry if it wasn’t as clearly written as it should have been.

      I’m glad the point of the post is obvious to you, but based on admittedly anecdotal evidence I think you’re in the minority.

  2. It’s a very important point that goes beyond species compensation in community ecology (although the basic misconception was the main reason we wrote this* and that* a few years ago).

    Any time there is spatial heterogeneity across population patches, the correlation in habitat quality is limited in the same way for >2 patches.

    Jeremy, I could probably dig up some reviewer reports (and at least one paper) to flesh out your anecdotes with ‘data’ if you like, it apparently isn’t as obvious to some people as it needs to be.

    Jim, I wonder if the problem here stems from the fact that you can have a very simple competitive community, where all species are otherwise identical at the within and between species levels (e.g., as in the ‘diffuse competition’ communities of Roughgarden & Hughes 2000), yet simply moving from a 2 to 3 species community (adding another ‘identical’ species) limits the potential correlation structure of species (or environmental) fluctuations. Some more confusion might arise from the differences in population vs per-capita growth fluctuations.

    * Some excellent, indirect journal plugging by our favourite (blogging) Oikos Ed ;o)

  3. Thank you Jeremy for your interesting post. However, I was wondering about some of your statements that is “… the larger s is, the less-negative the off-diagonal elements of the correlation matrix can possibly be, on average.” This statement apparently contradicts what has been argued about the compensatory mechanisms responsible to stabilize aggregate community properties as species richness increase.

    The work of Robert May (1972) and further reviews (Cottingham et al, 2001) have emphasized that as species richness increase, compensatory dynamics among populations also increase (correlations become more negative), and this is responsible for the stabilizing effect observed at the community level.

    Why this pattern is apparent? It is something that people working with diversity-stability relationship are not considering for?

    I will be glad if you or any other could do some comment about that.



    • Hi Adriano,

      When you write “as species richness increases, compensatory dynamics among populations also increase (correlations become more negative”, I think you’re articulating the widespread but mistaken view this post is pointing out. I think if you look carefully at both the references you cite, you’ll find that they *don’t* say this. I know May (1972) doesn’t. May (1972) actually has *nothing* to do with density compensation at all! May (1972) is about “stability” in a *very* different sense. To be precise, May (1972) is about how the real part of the largest eigenvalue of the community (Jacobian) matrix changes as a function of species richness and the fraction of non-zero matrix elements when the non-zero matrix elements are chosen randomly from a normal distribution with zero mean and a particular standard deviation. To be more informal, May (1972) is about how whether, and how quickly, a community returns to equilibrium following a small perturbation that changes species’ densities only slightly from their equilibrium values. None of which has *anything* to do with how those species’ densities covary. For instance, species could fluctuate synchronously or asynchronously as their densities return to equilibrium, but May’s results don’t tell you anything about this. And while I haven’t read Cottingham et al. 2001 in a while, and do think that some ideas in that article aren’t explained in the best way, I’m pretty sure that Cottingham et al. don’t make a flat-out mistake like claiming that, as species richness increases, the average pairwise correlation between two species becomes increasingly negative.

      The larger issue here is that the “diversity-stability” literature is unfortunately a mess. Lots of totally-unrelated ideas have all been lumped together under the heading of “diversity-stability”. Existence (feasibility) of equilibria, rates of return to equilibria, patterns of covariation, variability of aggregate variables like total biomass and primary productivity…It’s the kind of thing that begs for a blog post to sort out–except that it would probably take several posts! And even then I doubt they’d succeed. Back in the early 80s Stuart Pimm wrote what was at the time a very well-cited article, describing all the different senses of “stability” in the ecological literature and pointing out that they basically had nothing to do with one another. And here we are years later, and people are only more confused about than they were in the 80s. So when I complain about the imprecision of words and the very real confusion it causes, the diversity-stability literature is what I’m thinking of.

      Frankly, unless you know some mathematics you probably have little hope of making heads or tails of the diversity-stability literature. You’ll almost inevitably struggle to understand any bit of it, much less how the different bits relate (or in many cases, *don’t* relate) to one another, or even realize that they *are* different bits. And the worst part is, you may well *not even realize that you’re confused*. Mere words are simply too imprecise to do justice to the math, especially for readers who are just looking to “get the gist” of the math. Because the same words that “give you the gist” (i.e. a highly-simplified summary) of one bit of math can also be used to “give you the gist” *of some completely different and unrelated bit of math*.

      • To be more informal, May (1972) is about how whether, and how quickly, a community returns to equilibrium following a small perturbation that changes species’ densities only slightly from their equilibrium values. None of which has *anything* to do with how those species’ densities covary

        In fact, I think it´s easy to argue that the eigenvalues and Jacobian matrix are directly linked to covariance patterns in species fluctuations. At least, an awful lot of work in locally stable discrete time systems points in that direction, e.g. Ives et al (1999, Science; 2000, Ecology Letters) and the Hughes and Roughgarden paper I linked to above are good places to start with pretty simple community structures.

        A really excellent, more general, though probably underappreciated paper dealing with population covariance matrices in more complex systems is Greenman & Benton (2005, Theoretical Population Biology). It does a couple of odd things I still don´t fully understand, but really shows how the Jacobian links with the environmental covariance matrix to drive population covariances for locally stable and ´less´ stable systems, even in coloured environments!

        Of course, some mathematical knowledge is required to get the most out of these papers, but the earlier ones are certainly a good place to start.

      • Yes, I’m aware of those papers. A couple of technical points (which I know you know Mike, but probably many readers don’t). May’s results are for a deterministic system, and they don’t assume any particular model–they just assume that there is some (possibly nonlinear) model that one could write down that would have an interior equilibrium and an associated Jacobian with the sorts of properties May assumes that his Jacobians have. In contrast, results like those of Ives, Hughes, etc. are, if memory serves for stochastic Lotka-Volterra systems, i.e. a specific model is assumed.

        Technical quibbling aside, yes, you’re right that, in general, the ways in which species interact, as summarized by a Jacobian matrix, will have consequences for how they covary over time. I intentionally avoided saying this, because it would have undermined the point of the post. The point of the post is to force people to try to force people to think precisely about this stuff and actually get to grips with the math. Rather than just settling for verbal summaries of the math which are imprecise at best and seriously misleading at worst. If you tell people who are not mathematically inclined “there are close linkages between Jacobian matrices and the resulting patterns of correlation in species’ abundances”, what people hear is something like “Jacobian matrices and correlation matrices are basically the same”. And unfortunately, even if you follow up by telling people “Go read Ives, Hughes, and Benton to see what I mean”, most people who are not mathematically inclined will either not bother, or will read them and misunderstand them (or understand them only very superficially, which amounts to misunderstanding them). Wish I knew what to do about this, but I don’t.

        A depressing line of thought, I know. I guess I’m just grouchier than usual today.

      • most people who are not mathematically inclined will either not bother, or will read them and misunderstand them (or understand them only very superficially, which amounts to misunderstanding them). Wish I knew what to do about this, but I don’t.

        Make them required reading for some of the courses you’re teaching 😉

        Being really brave, one could build an entire course around Greenman & Benton (2005)!

        (this would probably make Ripa & Ives (2003, TPB) a prerequisite course)

    • p.s. Just to be clear Adriano, my reply is not at all meant as a criticism of you or anything you wrote. My reply is a lament that this literature is so difficult to make sense of, especially for anyone who doesn’t know a lot of math.

      • Thank you Jeremy for your elucidating reply! I totally agree with the confusion around the term stability means. My post was intended to bring into the discussion a subject that has been hotly debated in literature over the years and in my opinion have elements which are related to the message of your post.
        Thank you.

  4. Hrm. Does this apply for predator-prey interactions as well? Might this be a simpler explanation behind the whole weak-interactions-are-the-rule pattern seen in so many papers?

    I also wonder to what extent that those low correlations at high diversity (as that’s what we’re talking about here) could in part be driven by supression effects from the competitive network in and of itself.

    • Hi Jarrett,

      Not sure what you mean by “apply for predator-prey interactions as well”. But it applies to any correlation matrix. So for instance, if for some reason you were looking at the correlations in abundance among a whole bunch of predator and prey species (or a bunch of mutualists, or whatever), then the average pairwise correlation can’t possibly be much below zero. This is a mathematical fact about correlation matrices, which is *totally* independent of the underlying mechanisms generating the variability. If you have a matrix giving the correlations among a bunch of variables, that matrix will be positive semidefinite. Which means that, if your matrix has a whole bunch of variables, the average off-diagonal element of the matrix can’t *possibly* be more than slightly negative. Mike Fowler’s comment emphasizes this, and perhaps the post wasn’t as clear about this as it should’ve been. The post is about the relevance of this mathematical fact to compensatory dynamics, but it’s a fact about any correlation matrix, not just matrices of the correlations of species’ abundances.

      “I also wonder to what extent that those low correlations at high diversity (as that’s what we’re talking about here) could in part be driven by supression effects from the competitive network in and of itself.”

      Afraid I don’t know what you mean by “suppression effects”. Can you clarify? In any case, it’s not clear to me how you would tell if such “suppression effects,” whatever they might be, are operating in a many-species system. Since it’s mathematically impossible for the average pairwise correlation to be very negative in a many-species system, I’m not sure it’s even meaningful to talk about biological mechanisms that prevent the average pairwise correlation from being very negative in a many-species system. How can it be meaningful to talk about the cause of something that was impossible anyway? That’s like trying to talk about the cause of the fact that 2+2 does not equal 5. Maybe I’m misunderstanding what you mean here. You could talk about mechanisms that prevent *any given* pairwise correlation from being very negative (or very positive)–is that what you mean?

      Aside: do you think this fact about correlation matrices has any implications for the interpretation of structural equation models (which typically are based on correlation matrices, IIRC)? Perhaps not, I don’t know.

    • Re: the application to “weak interactions”, there isn’t one, unless you want to define “interactions” as “pairwise correlations”. Which I for one don’t! 😉

      There are various formal definitions of “interaction strength” in the theoretical literature. See Laska and Wootton’s 1998 paper in Ecology for a review. (And of course there are various informal and vague definitions in the heads of empiricists, which, as usual with informal and vague definitions, mostly leads to confusion.) But unless you’re *defining* interaction strengths as correlations (which no formal definition does), there’s no reason why a matrix of interaction strengths has to be positive semidefinite. Indeed, for most definitions of “interaction strength” of which I’m aware, an interaction strength matrix need not be any special kind of matrix at all. It need not be symmetrical around the diagonal, for instance, or have all the diagonal entries be identical, or etc. In principle, any element in the matrix can take on any real numbered value.

  5. […] comments on a previous post indicated some understandable confusion on the part of some commenters as to the relationship (or […]

  6. For folks who are interesting in reading more about this, there is a nice book chapter on it from a few years back:

    Brown JH, Bedrick EJ, Ernest SKM, Catron JE, Kelly JF. 2004. Constraints on negative relationships: mathematical causes and ecological consequences. in The nature of scientific evidence: empirical, statistical, and philosophical considerations. Pages 298-308.

    This chapter was part of what Jim Brown and I talked about while I was interviewing for a graduate position in his lab… in 1999. So, if you’re frustrated by journal turn around times, just remember that it can be a whole lot worse.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: