How to operationalise a complexity perspective is the big question,... it seems to me I have no simple solutions, but these are the directions I am interested in: 1. Pay attention to Ashby’s Law of Requisite Variety, as well as Occam’s Razor. I.e. the models we develop need to have a sufficient degree of complexity before they can adequately represent the world. For example, multiple conjunctural causation models. 2. Pay as much attention to real time monitoring as to episodic evaluations. In complex unpredictable settings we need more data not less 3. Pay as much attention to inductive thinking (pattern finding) about what has happened as to deductive thinking (hypotheses development and testing) about what will happen. Where the range of possible outcomes is big, we need useful search mechanisms (e.g. data mining algorithms), before focusing in with rigourous tests 4. Apply more eyeballs to the problem: make data sets publicly accessible wherever possible. So much survey data from development projects is woefully under-analysed and under-utilised. Eric Raymond famously claimed that “given enough eyeballs, all bugs are shallow.”

A Simple Measure of Success in a Complex World? Mission impossible. All what we can hope for is complex measure of complexity. This does not mean that the newly developed measure is 'hard to apply, undesrtstand...', it means that complex phenomenon can be simplified only in a complex way. More on this thesis: - http://www.sdeval.si/Publikacije-za-komisijo-za-vrednotenje/Meso-Matrical-Synthesis-of-the-Incommensurable.html - http://www.sdeval.si/Objave/Divided-we-stand.html

I agree about the complex world where we live in. I found this topic very interesting. Greetings!

@Rick Davies, how do you find Occam's razor relevant for dealing with complexity? OR is saying that reality is simple and so scientific tools that produce simple results can be taken as a proof of truth. I am also not convinced that big data is a road to better understanding complexity - since complexity is not microscopic issue but eqaully macroscopic - this is exactly the problem. Our proposal is to develop mesoscopic answer to this question.

I believe this is not the right question. Saying "we understand the world is complex" and then going on asking for the same things one always asked for is a contradiction. Understanding something means acting differently afterwards and realising that different questions become valuable and others become useless and counterproductive. But that is not usually what happens since that would mean questioning the rules of the game.

First of all I wish to thank you Caroline for your blog. I am following it only since recently but I immediately found it very thought provoking, opening cutting-edge topics (in my narrow focus) and offering excellent structuration of argument and commentaries. Your blog is one of the best I have ever happened upon. The challenge for simple presentation of complex world is in my view one of the most acute matters at present and visible to us exactly through a paradox mentioned in the title of your post. It is paradox because nobody really expects that complex world can be or should be presented in a simple measure. The challenge is not to kill complexity but to find out how to live with it taking into account limited cognitive capacities of humans as well as constrained institutional, regulatory and management possibilities. So the challenge seems to pose itself as a question how to simplify complex world without decomplexifying it with measurement, conceptualisation, communication, management. Or analogously, how to simplify complexity in a complex way. Which concept offers the simplest view of complexity? Very recently I have delightful opportunity to comment on one of Ramalingam's older papers (obtained via Patricia Rogers, Better Evaluation; Ramalingam B., H. Jones, T. Reba, J. Young. 2008. Exploring the science of complexity Ideas and implications for development and humanitarian efforts. London: Overseas Development Institute, Working Paper 285, 78 pp.). I certainly support their diagnosis of the challenge and fully share their aspirations. I found it very relevant for my study so I will certainly refer to it in my forthcoming book. My criticism is that their methodological aspect is mostly absent and so one can not see how theoretical results can be practically applied in evaluation of development and humanitarian efforts. This is probably just a consequence of one standard problem of studies which aim at defining operationally a concept of complexity. The theoretical aspect of their study is stretched between the system theory, theory of chaos and theory of complexity. But these need to be distinguished as three different 'theories of truth' (James W. 2002. Pragmatism: Will to believe, 1906, Lowell Institute; 1907, Columbia University). So they should be strictly shown not only as co-present, but also as separated. Some processes are primarily systemic, other are primarily chaotic and third may be mainly complex. Only the later is our concern, the concept of ordered complexity (Prigogine). If concept of complexity is too much complexified, it becomes disordered and chaotic and this largely means beyond our (non-scientific) comprehension. Let me illustrate with an example from the paper: 'The concept of phase space and attractors are central to understanding complexity, as complexity relates to specific kinds of system trajectories through phase space over time.' True, this centrality has been recognised for complex processes in nature. The problem is that ‘phase space’ and ‘attractors’ are not central for policy-makers, they don't apply these scientific concepts. Complex processes are specific for systems in transformation. For this reason, it is the most relevant to study transformative changes and transformative mechanisms, not attractors as fields of relative balance, stability and predictability (or resilience, for the same reason – vs. adaptability). Concept of phase space frames all hypothetical states of the system, but policy makers are usually concerned with very specific states of the system which are normally located in neighbourhood. Our challenge is probably not to develop the most exact copy of complexity science in nature and then translate it for explanation of social processes, but to find the simplest version of complex methodology, that describes the widest possible range of social processes. Next example: 'Accepting the notion of chaos … encourages an acknowledgement of the continual change in social systems'. As far as I understand the concept of chaos (Gleick), this is not really creative but very replicative process - scale invariant iteration of fractals. Creativity is usually attributed to evolutionary processes and even more to complex processes - as such, no interference of chaotic processes. Next example: 'Self-organisation is where macro-scale patterns of behaviour occur as the result of the interactions of individuals who act according to their own goals and aims and based on their limited information and perspective on the situation.' This is perfectly in line with Hollings, Giddens and mainstream science in general. What distinguishes complexity theory from theory of chaos is that the later develops with processes which maintain direct relationship between micro and macro level of the system – just like in the old style Newtonian paradigm of science, which is linear and where elements of the system are commensurable – aggregatable from micro to macro. On the other hand, complexity is essentially describing mesoscopic level processes (meso 1, 2, 3; as Dopfer, Potts, Foster, and meso 2a and 2b as I have proposed) – emergent mechanisms of qualitative change, selection, translation, intermediation between polar oppositions of micro and macro. I have tried to approach complex social systems from meso level of its structuration since it seems the most simple way of observe complex systems and much more potential in explanatory power. To *Rick Davies: I am somehow not convinced that dealing with complexity demands big data. Big data concept is basically meant for dealing with chaotic processes (see Cynefin model). Linking complexity to larger amount of data is linked to quantitative idea of complexity, which is reductionist and so wrong. Concept of complexity is paradigmatically different from the concept of simplicity, so the difference is first of all qualitative, unbridgeable with ever larger piles of data. The challenge is how to reorganise and reconceptualise available data to obtain complex picture. We already live in information society with gigatons of data available at hand or at least for not too many dollars which nevertheless remain rather poorly exploited. This can be proved with the absence of synthesis, since we live in very analytical age where each detail is terribly important. To *Caroline Heider: How do we define success from the outset? Complex measures of success are mesoscopic in nature, in my view: this means definition of success must be a set of partly compatible indicators, which are carefully integrated (with the help of mesoscopic algorithm of synthesis), to enable qualitative summative interpretation of the complex matter in holistic way. However, it still remains question on the nature of complexity measure – they are certainly not measures of effectiveness, efficiency, relevance. What are they? I believe answer will be context specific – depending on concrete evaluation situations. We have proposed measures of synergy – in one of our evaluations we developed three integral measures of integration – measure of cohesion and two measures of balance, strong and weak. What is the most important is to observe, that measures of complexity are mostly of hybrid (composite) nature, as they are obtained on the meso level of evaluation. I am pretty much convinced that the real challenge (How do we define success simply in complex operations?) is that mainstream thinking is micro or macro funded, while complexity demands mesoscopic reasoning. The answer is pretty simple, transforming logic which enables it is the real tuff thing in our shared challenge.

Re “How do we define meaningful targets at the interim that will both channel resources and work in the right direction to achieve these goals, and serve as a basis for evaluation?” Defining targets does not seem to have been a problem up to now :-(, but defining meaningful targets would presumably mean taking local context into account in the process, which could lead us in the direction of very customised targets. That solution would then present problems for a portfolio level evaluation that tried to make aggregate judgements about overall performance. [Unless performance was assessed in more meta terms e.g. as percentage of target value achieved] Alternately, a common across-the-board target could be retained if the evaluation process was able to include one or more weightings that took into account local contextual factors. This may not solve the problem because it could be argued that some contextual factors may be more important in some settings than others - so context factor X could not be given the same weighting across all settings :-(. There is another possible alternative that I have been interested in, which is pair comparisons, which allow detailed comparisons of individual cases, but not necessarily the same specific achievement criteria across all cases. Individual pair comparison judgements can be aggregated into overall performance rankings. Pair comparison is one means of aggregating votes and the analysis of their merits and limitations has been given some attention over the years.

Rick, I see your points about global goals and targets and translating them into something meaningful for specific country contexts or using country specific targets and finding ways to aggregate them up. But, maybe more so: high-level goals still need to be translated into outcomes that can be operationalized. Otherwise the leap from a program activity to a high-level goal, such as 3% poverty by 2030, might be too large, too distant.