Back in 2004, in the early days of community planning Scotregen asked the academic, Chik Collins to consider the prospects for more successful regeneration in future.

“If way to the better there be, it exacts a full look at the worse.” Thomas Hardy

From “Patchy” Achievements to Great Expectations

Let us begin with a quotation from a review carried out by Professor Michael Carley “in order to support the development of the Scottish Executive’s community regeneration statement, now published as: Better Communities in Scotland: Closing the Gap.”

Despite thirty years of effort, achievements of neighbourhood renewal are patchy. While there has been much physical regeneration, and some good examples of communities taking the lead in improving their neighbourhoods, there is little evidence of higher rates of social inclusion for the most disadvantaged households. Indeed, a striking finding is the degree of economic inactivity of households in deprived neighbourhoods, set against an increasing concentration of deprived households in social housing in deprived areas.

What it seems to be saying, bearing in mind the euphemistic nature of the term “patchy”, is that three decades worth of community regeneration policy has not done nearly enough to make things better, and that in some rather significant ways it has helped to make things worse – sometimes in and of itself, but more generally because of how it has interacted with wider policies.

This is not a controversial view. It is well known that a whole series of factors in recent decades has very significantly increased inequality – shifts in taxation; a fall in the relative level of most benefits and pensions; the ‘flexibilisation’ of the labour market, and the relative depression of wages for much unskilled work in both manufacturing and services. It is also well known that housing policy has worked to concentrate the poorest households in the least desirable areas of social housing, partly by rearranging subsidies so as, in effect, to drive out wage earners. Ongoing funding cuts have steadily undermined the already very limited capacity of local authorities to even begin to manage the effects.

The results of such policies have been predictable. They have provided the key bases for the development of increasingly imbalanced and troubled communities around our towns and cities. They have been at the root of a process of community degeneration. Attempts at regeneration have generally failed to address these problems, primarily (though by no means only) because they have never done enough to counter the underlying causes – and, realistically, separate from much wider policies, never could. In saying this we need to bear in mind that these attempts include some major, long-term, initiatives, led by central government itself – with substantial resources ultimately being diverted from other areas of at least comparable need to fund them. In the New Life programme some £485m was spent in just 4 areas over a 10-year period. We are not yet talking about the many areas which have never been the subject of such concerted action. But even in the main initiative areas themselves there have been notable failures to bring about improvements. Worse than the mere failure to make things better, however, there have been some highly undesirable outcomes. Not least of these has been the marked tendency for ‘partnership’ initiatives to undermine, if not destroy, some of those pre-existing community organisations which previously helped to maintain some kind of social fabric in problem areas.

The history of community regeneration in Scotland, then, is by no means a history of success. But now, it is being suggested, there are reasons for optimism – indeed great expectations of much better results. These expectations reflect the move to ‘Community Planning’ as heralded by the publication of Better Communities in Scotland. Henceforth, it is going to be primarily the job of local authorities to lead the Community Planning Partnerships which must deliver these much better results. Previous generations of partnerships, it is claimed, have provided firm foundations on which these new CPPs must be built. But what are these firm foundations? They can only be those foundations which have hitherto provided the basis for the delivery of those “patchy” results. They are foundations, moreover, which we apparently know to be firm despite the fact, as pointed out by Better Communities and other related documents, that we have never yet properly measured the success of local initiatives. However, now that the responsibility to deliver on the basis of these foundations is to be given decisively to the local authorities, performance is to be much more rigorously measured. The irony will not be lost on the reader.

In this light, if local authorities are going to have to lead the Community Planning process they might want to do some serious thinking. They might want to begin by thinking about the nature of what it is that they are going to be trying to lead. This might involve trying to grasp the way in which current conceptions and practices in community regeneration have developed over the period since 1988 – when the current model of partnership emerged in the New Life programme during Malcolm Rifkind’s reign at the Scottish Office. This might also help anyone who would want to ask how we arrived, and how we might act, in what might seem a rather perplexing situation.

From New Life to Community Planning: Policy, Implementation and Evaluation

The consensus that today supports the partnership approach to community regeneration belies its origins. Think back to late 1987. Malcolm Rifkind’s Scottish Office is trying to find a way to give practical effect to the principles of Thatcherism in a Scotland which has just voted decisively to reject them. He is arguing that the Scots as a nation suffer from a ‘dependency culture’, and that his Scottish Office will, notwithstanding the apparent absence of a legitimate mandate, set out to ‘roll back the state’ and fashion an ‘enterprise culture’. Other people are talking about the arrival of ‘the doomsday scenario’ and about the need for the Scottish people to have a much greater degree of power to resist such alien impositions.

The New Life programme, which is still at the root of the current notion of partnership, was not only central to Rifkind’s project, it was in many ways a pathbreaker for it. Its central aim was to take the Conservatives’ campaign against public sector housing in Scotland to a new level, and to establish this as the central feature of a new approach to Scotland’s urban problems – one which would be developed intensively in a small number of ‘partnership areas’ before being generalised much more widely. Scotland’s local authorities were themselves not to be trusted with the implementation of such a flagship project. Instead the Scottish Office itself would be responsible for leading the process, with the local authorities compelled to play a more junior “enabling role”.

Rifkind told us much of this very directly himself when he presented the Second Annual Lecture of SURF in 1999. Perhaps the most striking aspect of that lecture is the pleasure with which he recorded how the agenda he gave us in those days seemed, within a few years, to become a matter of consensus. In another context it might have been tantamount to rubbing salt in the wounds, but he could have expected to cause no more than some minor irritation to a few stalwarts. For, when the Conservatives were re-elected in 1992, partnership became, more than ever, the ‘only show in town’. Prior to that some had ‘played the game’ as a way of getting resources for ‘their own’ areas, but with some serious reservations. After 1992 people adapted themselves to what was increasingly seen as ‘a given’. By 1999, Rifkind was speaking in the knowledge that Donald Dewar had himself, in delivering the First Annual Lecture to SURF in 1998, as Rifkind put it: “provided the Forum with a detailed assessment of the very satisfactory progress” of the New Life partnerships. Rifkind was also speaking in the knowledge that the new Social Inclusion Partnerships were being developed firmly along the lines of the partnership model. Rifkind expressed great pride in his achievement.

It is unfortunate that Labour gave such a strong endorsement to the New Life programme before it had been evaluated. Had they waited for the New Life evaluation, and had they been willing to hold it up to the light a little more, then they would have found more than enough to encourage more circumspection. The results were no exception to the pattern identified by Carley – “patchy”. Indeed, avoiding euphemisms, and bearing in mind the original objectives and money allocated, the results were actually very poor. If these were the results of 10 years of concerted effort, and almost £500m of expenditure, then it should be no surprise that the SIPs themselves, trying to work to the same model and with very much less in the way of resources, seem to have struggled in a number of very significant ways.

But such outcomes are no longer going to be good enough. Where there has been a real change with the emergence of Community Planning is in the tone of the evaluation of the SIPs, and in the agenda that is being set for the future of monitoring and evaluation . This is very clear in the recent report to Communities Scotland by Cambridge Economic Associates. Here the criticisms of the SIPs, in which, of course, local authorities have tended to play a prominent role, are significantly stronger and more direct than we have been used to seeing. Readers may well be familiar with these criticisms. The SIPs, it is argued, have too often not really known what they have been about, and even when they have, they have tended not to have a very clear idea of how to be about it. Beyond the public sector – and not infrequently within it – partnership working has been much too limited. Community engagement is still not working, and the private sector is still not really present. The spend has not been bent, and often the SIPs have failed seriously to attempt to bend it. Practices in monitoring and evaluation have been seriously deficient. Even adjusting expectations downwards significantly from those originally set, the results, we are told, have still been ‘disappointing’.

At one level the realism one finds here is not unwelcome. But the manner and tone of its delivery may have been demoralising for many who have worked with the SIPs. For there is little sense in the evaluation that there may have been something wrong with the basic conception of the SIPs – not least that they were conceived on the basis of a fundamental mis-evaluation of the New Life programme. For the ‘partnership’ framework is to remain firmly in place. The process of “learning from the past” indicates, we are told, the need for “evolution not revolution” – evolution into CPPs, led by local authorities.

Instead, the emphasis in the report’s criticism of the SIPs is very much on failure at the level of implementation. The shift here is quite stark, and appears somewhat ominous for those who are to be charged with implementation henceforth. It is time, we are told, to confront those “matters which, if they are not dealt with in an appropriate manner, will strike at the heart of the regeneration process and the associated quest for social justice for deprived communities”. This will require that future evaluations exhibit “a ruthless recognition of the weaknesses which have obstructed previous initiatives in achieving their full aims and potentials”. This emphasis on heightened accountability for ‘delivery’ on central initiatives is certainly in line with government thinking more generally. The concern will be that with Community Planning being no more than an evolution in the basic ‘partnership’ approach, there might be a fair range of “weaknesses” to account for.

In this light, local authorities, and others who will be involved in implementation, will undoubtedly approach the Community Planning process with at least a degree of apprehension. They will be concerned about what will transpire when the “ruthless recognition of weakness” kicks in. The government will already have its explanation in place – unsatisfactory outcomes will be due to lack of proper co-operation between partners and a failure properly to “bend the spend”. There are clear enough precedents for the kind of ‘correctional’ measures that might follow. Teachers might be consulted on the possibilities.

To some, this whole scenario might seem rather perplexing, if not actually irrational. Why do we persevere with an approach to community regeneration which has failed to produce satisfactory outcomes over such a protracted period? Why would the Scottish Executive wish to saddle our local authorities with a statutory responsibility for Community Planning based on such an approach? Does the Executive want them to fail? Of course, politics and policy are not entirely rational, but that does not mean that we should not seek to make them more so – especially when the fate of some of our poorest communities is at stake.

Marcussen’s “ideational life cycle”.

Here the reader might indulge a short excursion into the field of political science. For the Danish political scientist Martin Marcussen would suggest that the tendency of policy ideas, like the idea of ‘partnership’, to become strongly resistant to rational criticism is not at all unique. It is, unfortunately, rather common. Policy ideas have a tendency to “thicken” and “harden” in an “ideational consensus”, and ultimately to become “consolidated in thick social institutions”. In so doing, they also become “sticky” –resistant to evidence of their deficiencies. Institutions, and the people in them, do things not because they work, but because they are right – right, that is, in terms of the prevailing (and “sticky”) ‘expert’ consensus about the ‘appropriate’ ideas in a particular field.

Such policy ideas emerge in the wake of external shocks or endogeneous political changes that render prior ideas “incapable of meeting the needs of political entrepreneurs”. There follows a period of uncertainty for peoplein the policy network – involving what social psychologists would call “cognitive dissonance”. How are they supposed to think about, and act in relation to, the problems they are supposed to be addressing in their work? Typically, people are keen to resolve such questions and re-establish some routine and normality.

The emergence of the new consensus that provides this is typically an elite-driven process. There is no requirement for it to be based on good ideas – never mind the best. What is important is that such ideas are powerfully disseminated. Marcussen identifies three kinds of dissemination which can work separately or in conjuction – “coercive” mechanisms stemming from political influence; “mimetic” mechanisms involving copying from others as a response to uncertainty, and “normative” mechanisms associated with professionalization and ‘good practice’. A crucial aspect of the dissemination process is that people within the policy network make a ‘psychological investment’ in the new ideas – as a resolution to the prior cognitive dissonance.

At this stage the process of institutionalisation of the policy idea is already under way: “The empirical challenge at this stage is … to locate these [new ideas] in new organizations or in old, but reformed, organizations”. Now the new policy idea becomes the definitive idea – something which is not open to question. People do not talk about alternatives, because ‘there is no alternative’. Adherence brings legitimacy, power and resources. But thinking differently becomes uncommon even for people whose motivations are less to do with material self-interest. They develop a “selective perception” – seeking out information that confirms their existing beliefs and attitudes, and ensures stability and routine. The system as a whole becomes resistant to challenge and change. It seems to require some new shock to the system to bring about any kind of change.

Scotland’s Partnership Cycle

If this is Marcussen’s theory of the “ideational life cycle”, then many readers, and perhaps especially those who have been around for a few years, will have already grasped its relevance to the emergence and development of the ‘partnership’ approach to community regeneration in Scotland. First came the ‘shock’ of ‘Thatcherism’, and especially that of the 1987 election result, followed by the dissemination of the partnership model – not necessarily a good idea for Scotland’s poorest areas, never mind the best – in the 1987-1992 period. This was linked to innovation and reorganisation on the institutional front – with the creation of Scottish Homes and Scottish Enterprise, and ultimately the reorganisation of local government itself. In this period the extent to which people actively ‘bought into’ the government’s thinking was, reflecting attitudes to ‘Thatcherism’ (not least among the Labour local authorities), still relatively limited. The form of the dissemination was, necessarily, largely coercive. After the return of the Major government in 1992, and the move to broaden and deepen the application of the ‘partnership’ model, people were increasingly under institutional, and also psychological, pressure to buy into it. Now the “mimetic” and “normative” dissemination mechanisms – in which SURF itself has had some significant role to play – grew in importance, and ‘partnership’ became increasingly thicker and institutionalised. By 1997 this was so much the case that the Labour landslide did not make a dramatic impact on the ideational cycle. Rather, the advent of the SIPs saw a further thickening and institutionalisation of the ‘partnership’ idea. More recently, the advent of Community Planning seems to represent a further stage in that same process. But it is also, perhaps, a ‘late’ stage – where failure is at least recognised, if wrongly blamed on the implementers. This, of course, is a way of ‘saving’ the central policy idea, but there is a limit to how often it can plausibly be invoked.

Those involved in the community regeneration policy network might take some comfort from the knowledge that the tendency of policy ideas to outlive their useful lives is not at all limited to their own policy domain. But they will take less comfort from knowing that had Marcussen wanted a case study with which to exemplify some of the most troubling implications of his “ideational life cycle”, then the idea of partnership in community regeneration here in Scotland might have been a candidate. It would be difficult to maintain that it was ever a ‘good’ idea – for it was founded on, and often actively promoted and developed, the very policies that were engendering community degeneration. And the continued resilience of the idea in the face of ongoing failures is striking. That it should have continued to prove resilient despite the Labour landslide of 1997 and the advent of the Scottish Parliament – both of which might have been expected to be candidates for the kind of ‘shock’ required to precipitate change – seems to make it all the more striking. After all, was one of the benefits of devolution not supposed to be a more open, deliberative and rational system of policy making – with committees developing, on the basis of proper scrutiny of all the evidence, policies more appropriate to the needs of Scottish society? This is certainly what was suggested. But in the field of community regeneration it seems not to have been the case.

A Space for Debate?

So, how would one expect the policy network, and in particular the key organizations in which the idea of partnership has been institutionalised, to respond to this argument? On the basis of Marcussen’s theory one would expect it to be, if at all possible, ignored altogether. ‘Just another whinge’, some might want to say. If that were not sufficient, and one would hope that in the light of the evidence it is not, then we might expect it to be summarily dismissed and marginalized. What kind of ‘rational’ person, it might be asked, can conceivably be opposed to partnerships and ‘joined-up working’ to try to bring about improvements for our poorest and most troubled communities? Well, we all know that no rational person is, or indeed could be, opposed to that. But the objection here is not, and never has been, to that formal aspect of ‘partnership’ policies. The objection is to their substantive, practical content – which reflects the continuation of the kinds of policies in relation to the distribution of income, patterns of employment, housing policy and local government funding which have created the problems that ‘partnerships’ are ostensibly supposed to address. Unfortunately, hitherto it has been possible for the policy makers to avoid this debate. And for so long as the evaluation processes failed sufficiently to clarify the reality of policy failure, the only people who really suffered from the effects of the failure to have the debate were those who were ostensibly supposed to benefit from ‘partnership’ – our poorest communities. But as we are entering the new phase of Community Planning it is clear that implementers will now also pay a price for recurrent failure – and they have rather more of a voice than poor communities. If nothing else, this will at least work to open up some kind of space for rational debate. Whether that space will be significant is yet to be determined – but local authorities in particular must try to ensure that it is.

For their part, local authorities, in conjunction with their local communities, must seek to develop, on the basis of a proper analysis of all of the evidence, their own critical appraisal of the evolution of the idea and practice of ‘partnership’. Their communities, and often their own employees, possess a rich and detailed knowledge of its limitations and failings which is waiting to be tapped. They must also begin to develop their own alternatives. In doing so they will be serving a number of related purposes. Firstly, they will be preparing themselves to respond to the criticism they will inevitably face when ‘partnership’ fails again. Secondly, they will be incubating ‘contenders’ for the next “ideational life cycle” which will emerge as the deficiencies of the ‘partnership’ model become ever more apparent – and hopefully they can help to ensure that the contending ideas are good ones. Thirdly, they will be trying to ensure that they still have the capacity and credibility to contribute in some meaningful way to the development and implementation of that new cycle – for if they do not now set out to do the first two things outlined above, then that credibility and that capacity may well not survive the centre’s “ruthless recognition of weaknesses”.

Conclusion: Arguing about Alternatives

Finally, what might the alternatives to ‘partnership’ look like? This is a vital question, of course. Failure to develop them will have serious consequences for the communities that the regeneration industry aims to serve. However, the absence of sufficiently developed alternatives is, especially in light of Marcussen’s argument, very much a predictable symptom of the dominance of an institutionally entrenched idea. Yet, however much people may feel the pull of the legitimacy and resources that its advocacy can bring, or to the routine and normality of familiar ways, this predictable lack of a sufficiently developed alternative is no credible argument for failing honestly to confront and recognise the deficiencies, if not irrationalities, of the prevailing approach. In fact it is no argument at all – and can look worryingly like an evasion. Transfer the response to some other spheres of human endeavour and this becomes clear. It could seem a bit like a DIY enthusiast replying to the suggestion that if he carries on with his current plan he might undermine the structural integrity of his house, by saying that he won’t stop until he has a sufficiently clear alternative approach on which basis to proceed.


Ultimately, we will all accept that the obligations we face to our poorest communities are irreducibly moral. If we are sincerely intent on “closing the gap” in Scotland, then we should bear in mind that this must begin from a full and proper recognition of the factors that have widened it, and the policies that have hitherto failed to narrow it. And it is from this process of rational recognition, which is very significantly lacking in Better Communities in Scotland, that the process of developing an alternative – for which we all have some responsibility – must begin.

There are those who will argue that this alternative must begin from a recognition that, while local authorities must play the lead role in community regeneration, they cannot realistically be expected to succeed with partnership initiatives where central government, mobilising much more significant resources, has itself failed. They will suggest that perhaps the most fundamental change required is that national policies that have done most to bring about community degeneration must be addressed – with housing policy being a key, but by no means the only, area for change. They will also argue for a properly funded expansion of public services, seeing this as a much more intelligent and economical use of resources than it is sometimes suggested to be. This would significantly augment the capacity of local authorities to ‘close the gap’ – both by increasing their capacity to address the needs of communities, and also in providing more stable and better paid jobs for some of the people who live in them. Along these lines, they would suggest, one can begin to imagine how local authorities could begin to lead a partnership process – though one of a significantly different kind – which might begin to deliver rather better results for local communities, and perhaps also significantly better relations with them. The present author has significant sympathy with such views. However, the priority at this stage is not yet to have the debate about the alternatives, but to open up a space where a rational debate about the need for them might take place.

Chik Collins

University of Paisley

June 2004