Finding a way to integrate standalone datascopes – a GloWORM

GloWORM – Global Web Observatory Resource Monitor

The Web Observatory is not a singular piece of software or a single repository but rather emerges as a function of accessibility to a set of distributed data telescopes (datascopes) that make their data and services available.
There are major challenges around automated discovery, content conversion, licensing and operational factors to deliver a *perfect* Web Observatory but to at least know what resources are available and what they address would be a great start.
There are currently hundreds of individual web pages,wiki’s or links to Open Data sets, standalone repositories, linked open data or to data sets that are privately shared as part of a project.
Nothing currently pulls all this together to allow visualisation / browsing by region, data-type, date range or topic. The ability to code or classify a dataset or tool by hand (crowd-sourcing the metadata) may be a much more pragmatic way of pulling together the resources in the short term so this should be considered where automated discovery or classification is not possible.
Challenges here include:

  • How to discover data vs static links
  • How to identify topics and QUICKLY enhance metadata through tags
  • How to combine public data with personal/project links
  • How to visualise/browse in an engaging way

This project will help to highlight the technical challenges in Observatory development and provide a user-level tool to show how the type/quantity of content changes over time.