Skills for development has been a neglected part of many bilateral and multilateral agencies’ work for the past quarter-century. In the late 1980s, the momentum for a renewed priority for primary education drew heavily on research from the World Bank that showed that rates of return to vocational education and training were comparatively low (e.g., Patrinos and Psacharopoulos 1994). This led to revisiting arguments that an academic education was the best vocational preparation (cf. Foster 1965) and attention to new evidence about the ineffectiveness of public vocational education and training (e.g., Middleton, Ziderman and Adams 1994). As a result, skills for development slid off the agenda of many agencies and remained off until the very recent past and discussions about a more integrated development vision under the Sustainable Development Goals.
Simon McGrath, Professor of International Education and Development, University of Nottingham, talks about transforming skills development in relation to two key opportunities in this HEART Talks, on Transforming skills development.
DFID was one such agency but skills are now back on its agenda. This means that one key question for DFID staff is: what kind of skills for development interventions might be most advisable? One way of exploring this question is to look at what kinds of approaches Britain has been exporting during this last quarter-century. In spite of DFID’s withdrawal from the field, Britain remains one of the most influential actors in skills for development, but through a range of other actors.
Much of what is present in the broader British international activity on skills can be classed as being part of the older tradition of technical assistance. Linked more to export activities, this focuses on strengthening public providers through equipment, staff upgrading, curriculum reform, management improvement, etc. However, it became hugely unpopular with Development Assistance Committee members by the 1990s as it did not appear to lead to sustainable improvement. Moreover, it also has strong connotations of tied aid.
However Britain has also been central to the most influential trend in skills policymaking of the last 20 years. Here the influence is quite diffuse and reflects a broader Anglophone trend. In the UK, it is exemplified by Scotland more than England to the extent that the Scottish Qualifications Authority (SQA) has been the key institution; but Australia, New Zealand and South Africa have also been key actors. This can be termed a governance turn. Here the focus is on getting the national skills system right by building national qualifications frameworks (NQFs) and authorities, national and sectoral skills councils; and improving funding and quality assurance regimes. As well as the work of the SQA, there is also significant British presence in organisations such as European Training Foundation and UNESCO, which are also advocating this approach, whilst British consultants play an important role in the dissemination of these ideas.
As skills for development moves up DFID’s agenda, the governance turn appears to be an attractive approach, reflecting both wider British expertise and broad congruence with DFID approaches to development. However, the purpose of this skills reading pack is to urge caution before and, if necessary, after adopting this approach.
Evidence is a major emphasis in DFID’s thinking and it is important to confront this as a serious problem in considering such a policy option and, indeed, in writing this skills reading pack. Whilst there is widespread advocacy for the skills governance reforms, there is a paucity of good quality evidence. It needs to be remembered that evidence on such systematic reforms is relatively challenging to obtain, and the nature of the reforms make them unsuitable for researching by certain approaches that are given higher weighting in DFID’s hierarchy of evidence types.
What we do not have on the skills governance approach is enough evidence that it works. However, neither do we have much robust evidence that is does not. Yet, the lack of evidence has not precluded elements of the approach being adopted in more than 100 countries in the last decade. This means that we cannot simply wait for robust evidence but must consider the implications of what we do know for any DFID funding decisions.
In what follows, I will draw on a small set of recent papers that raise questions about the approach. The power of these lies not so much in their publication status (a couple are conference papers) but in the kinds of evidence that they draw upon and the backgrounds of the authors. The texts presented come from the coordinator of an International Labour Organization (ILO) review of NQFs; the leaders of a UNESCO- Southern African Development Community (SADC) review of VET systems in Southern Africa; the former CEO of SQA, who has given policy advice to a number of national policy processes; and other former UK officials who have also been involved in widespread policy advice internationally.
What is presented below, thus, is not a rigorous systematic review of a large peer-reviewed literature but a deliberately provocative summary of a small, emerging literature that sits at the cusp of research, evaluation and policy work. It is intended to offer a caution against a trend to adopt an approach without either sufficient evidence that it works or a deep understanding of the major challenges that occur in its implementation.
Key readings
State of evidence – The abstract for this paper was subject to a blind peer review process for conference acceptance. The evidential basis for the paper is the authors’ professional experience as officials in the British system and, as consultants and advisors to a number of international governments, as well as a review of the existing international evidence.
Key messages – This paper reviews the case of sector skills councils (SSCs), a core element of the governance reform agenda. It draws on evidence from the UK experience, as well as recent work in Uganda and elsewhere. The authors review the argument that a productive approach to linking skills supply and demand requires a focus on generating and sustaining a high skills equilibrium, a situation which has been contrasted in the literature with the UK’s voluntarist tradition that is seen as having resulted in supply and demand reaching equilibrium but at low levels of skills quantity and quality. One of the tools used in the UK, and increasingly internationally, is the sector skills council, as a way of trying to generate a strong institutional response at an appropriate level of aggregation. This is done by bringing together employers and skills suppliers around the specifics of a sector and its economic potential.
In the UK, the SSCs have had a very variable performance, reflecting in part the very different traditions from which they come and the very different realities of sectors. Although still in place (though in a new configuration), SSCs are less apparently a core of the current government’s strategy than they were under Labour. Overall, the authors suggest that UK SSCs were typically most successful in those sectors that were already strong; and poor at affecting a turnaround in those with a weak history of collaboration. Thus, there is poor evidence from this case of them working to transform institutional arrangements. Of course, this will be an even greater challenge in DFID priority countries, where other conditions may also be against the creation of high skill equilibria.
Strong leadership also appears to be a key factor in success. Yet, again, leadership capacity is likely to be a major challenge in most priority countries. Success also depends on sustainable funding, but most sectors in priority countries lack sufficient scale to be good sources for internal funding such as through levy-grant mechanisms, and are unlikely to be major priorities for public funding. Equally, SSCs require good labour market information in order to perform well but this too is in short supply in many settings.
The paper argues that SSCs can work, and there are a number of good examples from middle income and transition economies, but stresses that they are not a simple solution. Crucially, the authors caution that it is important to understand why such structures and/or traditions of cooperation are not already present. Implicitly, the paper argues that there is too little thinking about a robust theory of change that both outlines how SSCs are supposed to work and identifies the likely main obstacles to their good performance.
State of evidence – The abstract for this paper was subject to a blind peer review process for conference acceptance. The evidential basis for the paper is both a review of the literature and the author’s professional experience as the first Chief Executive of the Scottish Qualifications Authority, as an advisor to a number of international governments and as a longstanding commentator on NQFs in academic and practitioner literatures.
Key messages – Tuck uses the lens of theories of change to ask what insights this might provide into two other key elements of the skills governance toolkit: training funds and NQFs, with a predominant focus on the latter. Whilst acknowledging that others see NQFs as inherently flawed, Tuck argues that they can be useful, if more attention is paid to how they are used.
One of the major issues is that NQFs are typically stronger as a vision of what an education and training system should look like than on a roadmap of how to get from the “bad” present to the “good” future: “the NQF enthusiasts who are leading national reforms think that it’s enough to espouse the vision of the exciting new education and training system that the NQF will usher in, rather than understanding the ‘institutional logics’ that are usually very different” (p23).
He also notes that one strength of NQFs-as-policy, that their language brings together various stakeholders through its ability to allow for different interpretations, is a problem for NQFs-in-practice in that it leads to uncertainty and divergence about issues of micro-design.
State of evidence – This is a refereed article in a well-regarded journal. It draws on work done for the ILO, led by the author, which reviewed national evidence from 16 countries on the state of NQF implementation. The project findings were validated at an international workshop with senior managers from national qualifications authorities, international agency staff and senior international academics. The report was published by the ILO.
Key messages – The paper notes the very limited evidence that exists for NQFs having had successful impact, yet also acknowledges the continued faith in them as an answer to failing skills systems.
In a similar vein to Todd and Woodgate, Allais notes that there has been relative success but that has to do with good initial starting conditions. Thus, Scotland, a country which already had “strong institutions, a relatively strong economy, and relatively high employment” (p 256), as well as a small population, succeeded. Moreover, its success also owed to developing its system incrementally, over a relatively long period of time. Yet, as Allais notes, the vast majority of those attempting NQFs have neither Scotland’s fundamentals nor the perceived time to implement incrementally.
Allais focuses strongly on the notion of NQFs as borrowed policies, pointing to the inherent problems in such borrowing. These include the perennial issue of how far context is important. In keeping with the other papers here, there is a strong sense that the reform process both sees lack of strong institutions as the problem that needs solving but also assumes that new institutions can be borrowed and unproblematically transplanted. She also notes that the borrowing tends to be of a snapshot of the system frozen in a policy or an institutional arrangement, rather than the necessary reading of the system as an evolutionary process.
As evolutionary, political processes, Allais sees NQFs as infused with power. This is an understanding that DFID has with its attention to theories of change and political economy analysis. However, as policies, NQFs are full of a language of consensus that suggests that all are winners. At the same time, at the level of implementation, they are replete with a highly technical language that effectively excludes most stakeholders from participation and democratic oversight.
State of evidence – This is a refereed article in a well-regarded journal. It draws on work done for UNESCO and SADC, led by the authors, which reviewed national evidence from SADC countries on the state of skills development systems. All national data was validated at national stakeholder workshops and the final report was approved by the SADC Ministers of Education and published by UNESCO.
Key messages – This paper is primarily concerned with raising a higher level question about the nature of knowledge and evidence used in planning, implementing and evaluating skills reforms. The paper notes the very high level of official optimism about the benefits about the governance reform process but the real lack of hard evidence across the region for actual implementation, let alone impact. It argues that this is hard-wired into the way that these governance reforms are understood in that they have an implicit theory of change that puts the emphasis on the promulgation of policy rather than matters of implementation, and which assumes that implementation and impact necessarily and unproblematically occur, a similar point to Tuck’s concerns about many NQFs.
The empirical findings of the larger study make it clear that SADC countries continue to struggle to implement the set of governance reforms and a large gap continues to exist between the governance level and the actual reform of provider institutions, employer involvement and improvements in learning and employment outcomes.
Further references
- Foster P. 1965. The vocational school fallacy in development planning. In Anderson, A., Bowman, M. (Eds.), Education and Economic Development. Aldine, Chicago.
- Middleton J, Ziderman A & Adams A. 1993. Skills for Productivity. Oxford University Press, NY.
- Patrinos AH and Psacharopoulos G. 1994. Returns to investment in education: A global update. World Development 22, 9, 1325-1343.
- Thompson S. 2016. Skills Topic Guide. HEART.
Questions to guide reading
The set of skills system governance reforms are an intended response to a set of problems in national skills systems that can be theorised either as a collection of market failures or as a weakness of institutional arrangements, according to the economic theory being used. As a theoretical account of a better alternative than current realities in most countries they are attractive. However, a series of questions need to be asked about how they operate in practice that can be couched in terms of their theory of change, as raised explicitly in Tuck’s paper.
- It is important to consider the state of national skills system and why they are the way they are. Basically, most skills systems in DFID priority countries experience major market failures; have weak institutional relationships, including low trust and little collaborative practice; are poorly funded; and lack strong leadership capacity. The proposed reforms are intended to overturn this but it is generally unclear how the transformation is to take place. Thus, in considering a potential intervention of this kind it is essential to ask: how can current weakness be overcome; what political processes are needed; and what are the likely obstacles?
- NQFs in particular assume that better availability of information is desirable. There are strong grounds for arguing this, both in terms of equitable outcomes for marginalised learners / workers and in terms of labour market efficiency. However, it is important to ask: do asymmetries of information benefit anyone? For instance, employers potentially benefit from workers and rival employers not being able to judge the full market value of individuals’ skills. Might fuller information, therefore, depress training volumes by increasing wage demands (hence, reducing the firm’s return on the training investment) or the danger of poaching? Where does power enter in to the analysis?
- The sector skills council case raises a further set of questions regarding the scope of interventions. Several countries, such as the UK and South Africa, have argued that SSCs are needed to bring about system-wide change and, therefore, should be introduced across all sectors at the same time. However, the evidence seems to suggest that performance is very uneven. Is it better to attempt system-wide SSC reforms, with the likelihood of uneven uptake, or to focus on the most promising sectors? Is it likely that successes in lead development sectors might have a demonstration effect that would generate employer buy-in from less dynamic sectors?
- Given we are talking here about reforms in complex systems, sequencing is crucial. Allais portrays Scotland as having succeeded by being able to implement incrementally. However, this was in an aid-free context. Within the constraints of development assistance, what is the right sequencing of interventions within a programme cycle or series of cycles?
- Given the limited evidence for the success of such interventions and their weak theories of change, is there a sufficient case for supporting them from a DFID perspective? If DFID are open to supporting such interventions, what information is required to ensure the selection of viable reforms to support? Part of the problem of judging success of such initiatives is the typical lack of clarity about what would constitute success. Therefore, it is important to ask in each: what would constitute acceptable outputs, outcomes and impact of any DFID intervention in this area?