The drive towards transparency

Written by Terry Blake, technical services director at TSOTransparency? Open Data? Linked Data? If you have been following government policy for the last 12 months or so you’ll have a reasonably strong idea of what I’m talking about. Originally driven by the Labour government and now embraced by the incumbent Tory-Lib Dem coalition, the push for government transparency in the form of open data could not be greater. Giving the British public a true picture of what information is being created, collated and stored amongst the walls of both central and local government promises to provide genuine transparency to the services funded by tax payers. The race to open up data is now on and taking place at an impressive rate. The questions remain though; how achievable is this initiative, what are the issues and benefits we all face in trying to meet the utopian view and how do we go about it?

From the beginning
Firstly, where did it all start? In early December 2009, then Prime Minister Gordon Brown, announced plans to open up UK government data which included: public services performance data, new transport data and geospatial data. Who better to head up this new initiative than Sir Tim Berners-Lee and Professor Nigel Shadbolt, hired back in June 2009, producing a beta version of what is now known as the website. This project was underway by September 2009 and since then has developed at quite a pace, launching as a single and easy to use online access point in January 2010, and recently growing to present over 3,500 data sets from all over government.
Now, under the new Prime Minister David Cameron, the drive has increased further. The Conservative ‘Building the Big Society’ paper was a clear message of continued support for opening up data, encouraging citizens, communities and local government to utilise this information so that they could become more empowered and help to build the Britain that they want. After only a few weeks in office, Cameron set out his commitment to open up government data in a letter to government departments and with some clear deadlines too. The new Public Sector Transparency Board, chaired by Francis Maude, appointed both Sir Tim Berners-Lee and Professor Nigel Shadbolt as board members along with Tom Steinberg, the founder of mysociety. This will help build on achievements already made by, ensuring that these leading experts continue the drive to open up data and set open data standards across the public sector. At the time of writing, the Transparency Board had just set out its draft principles including that public data will be published in reusable, machine-readable form, using open standards and following relevant recommendations of the World Wide Web Consortium.

An untapped resource
The developments over the past few months are really no surprise to those of us who work across central government departments, government agencies and local authorities and those who have been in ICT for the last decade or so. We’ve been aware of the semantic web from Tim Berners-Lee’s speeches back in 1999 and could more recently see the reasoning behind the need for linked data. “I have a dream for the Web [in which computers] become capable of analysing all the data on the Web – the content, links, and transactions between people and computers. A ‘semantic web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The ‘intelligent agents’ that people have touted for ages will finally materialise.” To create intelligent agents though, we needed data that could act intelligently and with that, slowly but surely, linked open data was born.
More recently in January 2010, Tim Berners-Lee told BBC News: “It’s such an untapped resource…government data is something we have already spent the money on... and when it is sitting there on a disk in somebody’s office it is wasted.” If we as individuals, communities, societies and nations are to progress, surely the obvious thing would be to take what was once static information residing either online or in files and utilise the World Wide Web as Tim Berners-Lee intended. So now, over 20 years after its introduction we seem to have evolved our thinking and the ability to really harness the web in a way that can truly benefit us all. But how do we achieve this?
There have been two phases to this initiative so far. The initial push was to put up the data that already existed in whatever format was readily available, but that’s really not enough. In order for that data to be reusable it needs to be unlocked from the usual PDF, Excel and HTML formats and converted into linked, machine readable formats. The second step, the one we’re in now, requires organisations to adopt a series of standards when publishing their data using recommended linked data formats such as RDF. If we’re to build true transparency then data needs to be structured in a way that enables it to be correctly open and interoperable. New data can be captured in ways which makes it easy to transform it into linked data but for existing data your only option may be to retrofit it using advanced techniques to automatically structure and enrich content.
Thankfully many organisations, including ours, realise the importance of setting out best practice to provide a sustainable environment to publish data. TSO is working with organisations including the Cabinet Office, the COI and The National Archives to bring this together, establishing the principles for publishing government linked data and helping different people, organisations and departments see exactly what they need to do to deliver upon the open data drive. For instance, reference data sets for common data such as government departments and MPs all need to be created and shared so that all can refer to data in the same way. I can’t emphasis enough how important it is for people to work to a standard to ensure this all pulls together, which will mean that time and funds are not wasted and that everything works as a whole. Finally, making it easy for that data to be accessed through APIs will help to ensure it gets reused.

Of course, like Professor Nigel Shadbolt commented earlier this year: “Public bodies have a public duty to publish public data.” You may be wondering what can be done with all this data once it’s been published as linked data? How do the public use it to better empower their decisions on an individual or community basis? It’s fair to assume that the general public won’t access the data from and will need the data re-presented in a more user-friendly form. This is where projects such as TSO’s OpenUp challenge, and the 4iP fund, come in. These are simple competitions, projects and challenges that will help educate and encourage people to interact with the data in a fun but meaningful way where they can see the true value of it.

What’s happening overseas?
It’s also important to note that we’re not the only country embracing open data. The Obama administration launched to offer feeds from various departments early last year and other countries around the world are taking steps to partake in this greater transparency. It appears that finally government and society have evolved to generate something for the greater good, to make all our lives simpler, better informed and above all, to help us all become more responsible citizens. We all expect it, we can now all demand it and if we take the right approach we can deliver it.About TSO
TSO (The Stationery Office) is the leading provider of information management and publishing solutions to the public sector. We are the largest publisher by volume in the UK, publishing more than 8,000 titles a year in print and digital formats. Our experts help to create, structure, capture, transform and deliver some of the most important government information. TSO provides services, consultancy and infrastructure to deliver all aspects of the information lifecycle to the highest standards for our clients.
TSO has been at the forefront of working with public sector clients to open up published data. We create tools and processes to allow data to be created in a structured way; enrich data using text engineering techniques; convert data into formats to publish as linked data on the web and provide and host web environments that allow both humans and machines to access the data.
Privatised from HMSO (now The National Archives) in 1996, TSO was acquired in 2007 by Williams Lea, the leading global provider of Corporate Information Solutions.

For more information
Visit and or follow on twitter,

Please register to comment on this article