DATAMI

The DATAMI Interface

Mathieu d'Aquin — Thu, 29 Mar 2012 08:58:15 +0000

As described in the previous post, one of the advantages of relying on flexible semantic technologies is that access to the information only requires querying the knowledge base that was built out of monitoring the web traffic of the user (with the datami-proxy) and processing the corresponding resources to be semantically annotated (with the datami text processing component). That makes it possible for use the build an interface entirely in HTML/Javascript, that focuses on providing appropriate visualization and interaction mechanisms to the user.

The current version of this interface has now been released in our code base (with the associated short documentation). At the moment, it looks like the screenshot below. Compared the the version in the demo video, the ability to define filters based on the date had been added. It is now possible to ask about the popular entities that I encountered “last week” for example.

datami-interface screenshot

The interaction with this interface is reasonably straightforward: entities and websites are shown in tag clouds. Hovering over one of them would show, in a tool-tip, more information about it and clicking on it would add a filter. Using the calendar at the top, the two boundaries of a range of dates can also be selected as filter. Hovering over the type of entities in one of the three entity clouds allows to select the type to display, depending on the entities available with the current filters.

In terms of development, the core of the interface is showing the results of SPARQL queries, similar to the one described before on this blog, adding to them particular patterns to reflect the current filters. Additional work is now required, especially not make the interface more appealing to users.

Querying my semantic personal web history knowledge base

Mathieu d'Aquin — Wed, 28 Mar 2012 22:28:12 +0000

The beauty of the DATAMI application is that, through the combination of the datami-proxy and the datami text processing component, what is being built is a complete knowledge base of what the user, me, encounters on the Web. The architecture of the application shows an additional component to score and rank entities from this knowledge base. In reality, the interface (as in the demo video) only needs to send the right queries to the knowledge base (as materialized in a SPARQL 1.1 compliant triple store) to obtain scored and ranked entities.

So let’s look at what sort of queries we need to get, say, the 25 most popular places in my web history.

The datami-proxy gives me indication about the websites that have been accessed, through requests. So basically, if I want to know which websites I have accessed, I need to query the ‘?ws’ such that

        ?r  ?ws

?r representing the request.

The annotation produced by the Stanbol Enhancer Service are then connected to the requests through a “related to” relation. So, if I wanted to know about all the entities ?x I encountered in websites, I would need to add:

        ?r  ?ea.
        ?ea  ?x

using the stucture of entity annotation returned by the enhancer service (i.e. the “entity reference” relation). Adding to it that the type of the entity should be “place”:

?ea

I get (as ?x) all the places I encountered through my online activities.

Now what we want is the 25 “most popular” ones. Starting simple, we can use as a score for popularity the number of websites mentioning the entity, so the query that would obtain the 25 most mentioned one (in SPARQL 1.1) would be:

select distinct ?x (count(distinct ?ws) as ?nws)
where {
   ?r  ?ea. 
   ?ea  ?x.
   ?r  ?ws.
   ?ea  
} group by ?x order by desc(?nws) limit 25

Easy!

Adding a tiny bit of complexity, we can then also retrieve the label of the entity and use the confidence returned by the enhancer service as part of the score. The query then becomes:

select distinct ?x ?l (count(distinct ?nws) * avg(?conf) as ?score)
where {
   ?r  ?ea.
   ?ea  ?x.
   ?r  ?ws.
   ?ea  ?conf.
   ?ea  ?l.
   ?ea  
} group by ?x ?l order by desc(?score) limit 25

And that’s done! All that is left to add to get the DATAMI interface is to add filters for selected entities, websites and times, and the possibility to change the type of the entities considered.

The DATAMI text processing component

Mathieu d'Aquin — Tue, 27 Mar 2012 15:42:26 +0000

Following the release of the datami-proxy, the next piece of the architecture of the DATAMI application to be made available is its text processing component. To a large extent, this is where the ‘magic’ of the application happens: this tool is the part of the architecture in charge of identifying important/significant entities within the elements of the user’s web history. This is where we use Apache Stanbol, developed in the IKS project; specifically taking benefit from the included enhancer service.

Since the Stanbol enhancer service is doing most of the work for us, the component is actually quite simple (see the source code and short documentation on our code base). It basically queries the triple store populated by the datami-proxy to obtain new texts to process, call the enhancer service from Stanbol and store the resulting annotations into another triple store (in connection with the information about the web interactions that led to the text).

The results however are very interesting: as the datami-proxy collects simple data about online resources and the text processing component enhances it with information about the semantic entities they contain, what is being built and constantly updated is a knowledge base of the things I encounter online.

DATAMI Initial Demo

Mathieu d'Aquin — Wed, 21 Mar 2012 18:42:34 +0000

We have been very busy in the last couple of weeks setting up an initial interface and demo: a first demonstrator of what DATAMI will be. This is clearly still far from the final product as we envisage it, but it shows the potential of the idea: background processes collecting and processing information about my online activities to provide me with a personal, semantic web history dashboard. But, since a video is better than I don’t really know how many words, here it is:

All the data on which this demo relies has been collected using the datami-proxy running on my personal laptop. It is then transparently processed (in batch running every hour) through the stanbol enhancer services to identify entities in the texts of the websites I encountered, as well as additional information about these entities. All these data are kept in a semantic store, which is being directly queried by the javascript interface. Many improvements are still needed before DATAMI could be made accessible, some of them more tricky then others:

Adding a way to explore and filter by ranges of dates
Adding information about the connections between the displayed entities
Putting some design into the interface… it is rather ugly and in need of a better look and feel
Improving data processing and querying to make it faster, and so that it can fit entirely on a standard laptop. This will require in particular to implement clever data trimming and caching mechanisms
Improving the entity ranking mechanism. At the moment, we use a combination of the popularity of the entity (how many websites mention it) and of the confidence with which Stanbol identifies it. However, Stanbol sometimes come up with rather funny results, and more sophisticated approaches might be needed to separate the important entities from the irrelevant ones.

The next few posts will be dedicated to the implementation of the components of this demo, and to updates regarding the progress on these improvements. So, watch this space!

First version of the datami-proxy

Mathieu d'Aquin — Thu, 08 Mar 2012 11:29:58 +0000

As shown in our initial Architecture Diagram, at the basis of the DATAMI application is a mechanism to record the Web traffic generated by a user, to generate a “Web History Store”, containing enough information for the IKS Stanbol services to do their “enrichment” magic. This mechanism is essentially based on a “Local Logging Proxy”, called the datami-proxy, and the first operational version of this tool is now available in our code base.

How does it work?

The datami-proxy is essentially (like most Web proxies) a Web server that, instead of directly delivering content, captures requests to other websites and redirects them to their intended destination. There can be a lot of different reasons to use a proxy, but in our case, the goal is to log information about the corresponding Web traffic. In other terms, this tool will monitor all the Web traffic coming in and out of the computer, and store (in a semantic triple store) information about the websites accessed, the time and the textual content of the responses. The aim is, in the next component of the architecture, to annotate this textual content in order to obtain a semantically enriched personal Web history.
In practice, the datami-proxy runs as a background process, alongside a semantic triple-store. The computer/browser of the user needs to be configured so that the datami-proxy is used as Web proxy for all HTTP requests. A wiki page in the code base provides all the instructions necessary to install, configure and run the datami-proxy on a user’s computer. Once configured, the datami-proxy should in principle run transparently for the user, not affecting their Web experience.

Tests, limitations and next steps

While the current version of the datami-proxy is only a development version, and many issues still need to be fixed, our overall experience with it is very positive. I have personally been running it on my computer for more than a month now, and I’m quite amazed about how, without me noticing, a knowledge base of everything I’m doing on the Web is slowly building up on my hard-drive (and being synced at regular intervals with an external store). In other terms, I now have the supreme power to query my own Web history for all sort of crazy things (how many google searches do I do per day? How many of the websites I have visited mention my name? At what time of the day am I the most active on the Web?). Soon, thanks to semantic enhancement, I will actually have an interface to do that, which will connect the different entities I encounter on the Web which each other.
Of course, as an “in development” system, there are still quite a few issues to be solved. For example, the datami-proxy does not handle cookies and sessions very well, meaning that I often have trouble authenticating to online websites (e.g., this blog). I currently tackle that by keeping a browser not configured to use the datami-proxy, and where I do all the things that the datami-proxy can’t handle. Besides tackling the other components of the DATAMI architecture, the continuous development and improvement of the tool will mean that, soon, we should obtain a more robust version that would seamlessly handle any type of Web traffic.

Initial Architecture Diagram

Mathieu d'Aquin — Thu, 02 Feb 2012 12:42:49 +0000

The nice thing about a project like DATAMI, which is mostly being developed from scratch, is that we always get to start with a kind of architecture diagram. In my view, while it does not really get into the details of what’s going on, this is still the best way to do project planning: identify the individual components and tasks, the dependencies between them, and their requirements. We made one on a whiteboard a couple of weeks ago with Carlo and I though that translating it into a human readable form (turns out the DATAMI team, i.e., Carlo and I, are not exactly human) would be the right first ‘serious’ post here.

Architecture overview of DATAMI

I won’t go into the details of it now, but the basic principle is that there are three main activities: recording, enhancing and visualising. Recording corresponds to capturing the online activities of the user, through a local web proxy. This tool already exists in an initial form and will be finalised soon. Enhancing is where the IKS Stanbol services are being used, extracting additional semantic information related to the online resources being encountered by the user through their online activities. Visualising is the last bit of the puzzle, where we will need to find a way to enable users to view, understand and explore their semantically enhanced web history. The whole data management and communication aspect of this architecture will be handled through the use of semantic technologies, with information being stored in RDF triple stores (Fuseki) and the exchange of information between components being tackled using the SPARQL and SPARQL-Update protocols.

Welcome to DATAMI

Mathieu d'Aquin — Thu, 26 Jan 2012 17:01:38 +0000

The DATAMI project/application intends to support users in managing their interactions with content on the Web. More and more information is being exchanged between individual users and a large variety of online organizations. From the point of view of the Web user, these exchanges are happening in a fragmented, un-managed way, which makes it harder for them to take full benefit from this content, to obtain an overview of their online activities and make efficient reuse of these activities. We will build on the Stanbol Services to produce a semantic personal Web history for users, enriched with information about the various types of entities (websites, people, organisations, places) they encounter. The need for such types of application relate in particular to scenarios where individuals rely on information found on the Web, but without necessarily having at their disposal the tools, time or capacity to organise the online content they interact with. In other terms, what DATAMI intends to do is to provide ways to automatically, without affecting the user’s normal activities, monitor and organise the resources he/she is interacting with, so that it can be later explored, re-used and retrieved.