An enabling environment for evidence-informed policy making
avatar

Question – what exactly does an evidence-informed policy look like (and who should be able to recognise it as such)?

The idea that policy should be informed by evidence and not based solely on it recognises that policy makers consider other factors beyond research when making decisions.  In this model, policy makers may consider and choose to ignore research evidence in response to factors such as political expediency, timing and resource constraints.  This makes it difficult to identify a policy that is actually informed by evidence when evidence is considered and rejected.   In fact it is probably easier to spot one that blatantly ignores a body of knowledge rather than one that considers and chooses to respond selectively to research evidence.

Two recent stories in the UK made me think about the concept of an enabling environment for evidence-informed policy making.  The education secretary was recently shown to have used unreliable polling data designed for commercial marketing purposes as evidence of the decline in the country’s education system.  Another minister was shown to be misrepresenting statistics from his own department to back up claims about the efficacy of his policies.

While on the face of it these two examples show that politicians’ will always find ways to use data to their advantage the fact that the stories are being discussed in the press highlight what I think of as components of an enabling environment for evidence informed policy making (in no particular order):

  • Public appetite – In both cases the policy makers felt the need to back up their statements with some data thus recognising a public appetite for evidence
  • Engaged citizens – In the case of the education secretary the dodgy polls were exposed by a freedom of information request by a retired teacher not by the media or opposition
  • Access to information
  • Processes and institutions that scrutinise use of evidence – in the second case the statistics authority wrote an open letter to the minister chastising his department for their poor use of evidence and forwarded a copy to the parliamentary committee with oversight of the department.
  • Media that hold policy makers to account.

The idea of an enabling environment is gaining traction and is one that INASP supports in our programmes through building capacity for access, production and use of research.   A recent post on from poverty to power looking at ‘How to plan when you don’t know what is going to happen’ suggests working on supporting an enabling environment rather than specific projects as part of suggestions on how aid should change.  The second post is also worth look here.

The points above are just a few components that spring to mind.  I would like to hear your thoughts on other components of an enabling environment for EIPM (and how to support them) and indeed what an evidence informed policy looks like (and who should recognise it as such).

Tagged , 2 Comments

MOOCs and educational development: Part 2
avatar

This is the second in a series of blog posts on MOOCs (massive open online courses).

Early in 2012, MIT announced its first MOOC. This was about electrical circuits. My academic background is in electrical engineering, so I was inspired to sign up and take a look. The first lecture was a lesson in humility. I realized I knew very little in spite of having a master’s degree in the discipline. I worked as a traditional engineer only for a few months after my studies and many years had passed since then, so I consoled myself after dropping out. It was MIT material, after all. Maybe there would be a more suitable MOOC for me later on.

In October 2012, I came across a tweet about an edX course in biostatistics and epidemiology, offered by Harvard University. The full title of the course was “PH207x Health in Numbers: Quantitative Methods in Clinical & Public Health Research.” I was looking to learn about statistics, and my work in life sciences editing and teaching scientific writing had given me an appetite for learning about the health sciences in a formal way. And the Harvard part was definitely a magnet.

The course was to start that day, and I signed up. It was going to be a 12-week course, and students were expected to spend about 10 hours a week on it. The course seemed interesting enough, but I doubted whether I would be able to spend that much time every week for nearly 3 months. I had just relocated to Mumbai and was settling into a new work routine.

The topics covered in the first 3 weeks were not difficult: basic statistics, probability, an introduction to Stata (a software application for statistical analysis), and concepts of prevalence and incidence. Every week, I watched a series of lectures given by two professors, Marcello Pagano and Fran Cook, both from the Harvard School of Public Health.

Here are a couple of screenshots from the lectures:

The monitor in the first image allowed handwritten input from a marker, which was overlaid on the slide text. The teachers would regularly walk to it and emphasize or write something. The second image shows this more clearly. The video had a number of options, such as turning on captions (a transcript of the speech), playing in HD (I didn’t use this – the regular mode was good enough), and changing the speed (0.75x, 1x, 1.25x, and 1.5x). For me 1.25x worked best, and I imagine 0.75x might have been good for people with English as a foreign language.

The videos were hosted on YouTube, so the server-side bandwidth couldn’t be faulted. But this is a topic for a later post.

Coming back to the nature of the videos, these were not made during classroom teaching. They were made for the MOOC. With the professor facing the camera, the excellent video-editing (showing close-up shots of the slides at key moments), the video controls (pausing, speed varying, captions), the experience was far better than sitting in a classroom. I’ll hasten to add that I’m not talking about small-group, discussion-oriented classes, but about large-group, lecture-driven classes. But really, how many people have the privilege of going through university education with most classes in the first format?

There were two kinds of continuous assessment: questions in between lectures, and weekly quizzes. Because of the quantitative and objective nature of the course material, all the questions were of the multiple-choice or fill in the blanks format.

The above screenshot shows a set of questions in between lectures (see the top bar which gives an idea of the mix of lecture videos and questions). Students could attempt these questions as many times as they wished until they got the answer right. But each quiz question could be attempted at most 10 times. I found this limit quite generous, except in a few cases where I just couldn’t figure out how to solve a problem and I found myself making wild guesses after a few attempts.

The course was a riveting experience the very first week. I met the weekly deadlines for quizzes for 3 weeks and did well. Then the material started becoming a little challenging. I began to worry. Twelve weeks seemed awfully long. I was going to take a 2-week vacation in December, and the course was to end in mid-January. And there was going to be a final exam worth 60% of the grade! Could I keep up?

To be continued…

See ‘MOOCs and educational development: Part 1

Tagged Leave a comment

MOOCs and educational development: Part 1
avatar

This is the first in a series of blog posts on MOOCs (massive open online courses).

In 2012, MOOCs gained so much prominence that they were written about in The Economist and The New York Times. Looking at what MOOCs potentially offer, it’s easy to see why they have taken the world of education by storm:

  • MOOCs are free. (At least this is true for the vast majority of MOOCs now.)
  • MOOCs are taught by real professors from top universities.
  • The short videos used in MOOCs are said to be inspired by TED talks. They’re not anything like the videos of classroom lectures that have existed for quite some time. A 40-minute lecture can be boring even when you’re sitting in the classroom. Record that and you might get something that no-one would want to see! But the videos in many MOOCs are made only for those MOOCs. They’re not byproducts of classroom courses.
  • Assignments, problem sets, and exams are not easy. A lot of e-learning content and courses out there mollycoddle learners. Not MOOCs. You have to be prepared to be challenged and even to fail. Most MOOCs have completion rates of around 10%.
  • You get a really nice certificate if you pass the course. When you put in something like 10 hours a week for 12 weeks and pass a nerve-wracking final exam, a certificate doesn’t seem so trivial.

MOOCs do not usually carry formal course credits, but this is changing with some MOOC providers tying up with assessment centers. For the time being, MOOCs are largely about putting yourself through an intensive learning experience just because you want to learn something. Continue reading “MOOCs and educational development: Part 1” »

Tagged 6 Comments

The state of art in ‘Monitoring and Evaluation’ and ‘Impact Evaluation’ practices
avatar

Reflections on the UK Evaluation Society Annual Conference (2013)

The UK Evaluation Society (UKES) Annual Conference is a unique opportunity to share knowledge and experience with other Monitoring and Evaluation (M&E) and Impact Evaluation (IE) professionals. Three main points for reflection inspired by the two-days of presentations and debates of this year’s edition that I would like to highlight here:

  1. The definition of Value For Money (VFM), that during a session facilitated by Save The Children was broken down into economic vs. operational VFM — the former as economic benefit coming from the implementation of the programme, the latter as the intrinsic value of the service (often having a public nature) provided by the activity (having better quality education is good per se, not dependent on the cost of provision);
  2. The IE community now is ready to leave behind the (sterile?) debate qualitative vs. quantitative methodologies for IE – a theory based mixed approach is considered to tacke more efficiently with the complexity of society; also, different methodologies serve different proposes — it is up to the evaluator to select the ‘right one’ each time;  
  3. The challenge that M&E and IE specialists encounter not to be considered as a threat but a resource by the organisations they work in; this is a reciprocal learning process, in which the M&E specialist deeply understands rationale, complexities, assumptions and risks of the activities and the programme managers use rigorously tested practices to improve delivery and operational and economic VFM. 

The Conference was also an opportunity to present INASP‘s approach to ensure both that IEs are carried out in a rigorous way and that the learning is shared with and informs the activity of relevant stakeholders. Continue reading “The state of art in ‘Monitoring and Evaluation’ and ‘Impact Evaluation’ practices” »

Tagged , , , , , , , , , 2 Comments

Publishers for Development updates and hold the dates!
avatar

Interest in Publishers for Development (PfD) continues to grow amongst our publishing partners and the wider community. Face-to-face networking is such an important part of our work so we have just posted information about our 2013 conference which will be held in Charles Darwin House, London on October 15th. This year the title we’ve chosen is ‘Forward Thinking: Developing a global research cycle which fully engages south and north‘ we are still finalizing the conference programme but it will have the following themes:

  • The changing landscape of availability (revisited)
  • Southern publishing: why we need to ensure its sustainability
  • Publishers for Development in Action – Awareness, Access and Use
  • What we measure matters… but what we measure isn’t all that matters
  • Advocacy is everything

Continue reading “Publishers for Development updates and hold the dates!” »

Tagged , , , Leave a comment

AuthorAID in Sri Lanka: Workshop and annual meeting
avatar

AuthorAID project team at Galle Face Hotel, Colombo. Clockwise from top left: Ravi Murugesan, Sioux Cumming, Julie Walker, Barbara Gastel, and Julie Brittain.

Last month I was in Colombo, Sri Lanka, to facilitate a train-the-trainers workshop on teaching research writing online. I think that has to be the narrowest topic I’ve ever taught! But the workshop turned out to be more than that.

Towards the end of 2012, I facilitated two online courses in research writing over AuthorAID Moodle, our e-learning platform. These courses were taken by nearly 70 researchers from about 20 developing countries. The high completion rate (~85%) and participation (~300 forum posts in each course) assured us that online courses can work in our context. Continue reading “AuthorAID in Sri Lanka: Workshop and annual meeting” »

Tagged , , Leave a comment