User login

Browsing Data (specific instructions for standalone demo)

The following instructions are particulary targeted to the standalone demo version. They apply, with minor modifications, to the live platform.

Introduction

The DAE platform was designed to provide a set of tools that would allow higher quality experimental research in Machine Perception, and Document Analysis in particular (cf. DARE paradigm). In order to make it appealing for the community to adopt, the platform offers some basic functions that hide its inner structures and functioning, but offer convenient tools for conducting reproducible experimental research.

Browsing

Data Browsing Pane

One of the platform's most elementary features is to provide access to experimental data. Although, eventually these data can be downloaded as large archives, there is an advantage in leveraging the fact that all data items are stored as individual parts. The "Browse Data" section in the left hand pane illustrates this.

By default, the interface displays the list of data sets available on the platform, as well as a selection of the documents it contains, as shown in the screenshot on the right.

The following interactions are possible:

  • Clicking on the data set title (e.g. "Unofficial GREC 2011 ...") will display the contents of this data set. Data sets can be organized hierarchically, containing other data sets.
  • Clicking on one of the example images will directly display the details of that specific document. The interface for accessing individual documents will be detailed later.
  • Clicking on the grey icon on the left hand side will download the entire data set. Note that all data items can have Copyright information attached. Downloading data triggers a Copyright agreement pop-up.
  • The banner on the upper side of the browsing pane allows to select various display options:
    • "New" - "Top Rated" - "Popular" order the list of available data according to their date of creation (newest on top) for "New", their rating (number of stars rated by users) for "Top Rated" or to the number of times they have been accessed or downloaded over the last period of time for "Popular".
    • "All" - "Datasets" - "Root Datasets" - "Images" select what kind of data types are displayed. By default, only "Root Datasets" are provided to the user. These are well identified data collections, possibly containing other data sets, and containing images. Users also can elect to display all stored data, all datasets or all images.
  • The "Search Names and Tags" pull down menu allows to search through the available data by either their name, or their associated tags.

Individual Document Interactions

The browsing features explained in the previous section allow the user to navigate through the various available hierarchies of data sets. However, when selecting a particular document image a series of other interactions are made available though the interface.Detailed Document Browsing

Note:Looking glass

Depending on browser, OS and JavaScript versions, thumbnails of document images may not display immediately. Reloading the page usually solves the problem. Clicking on the looking glass icon (when available) may also result in the image displaying correctly.

Documents can take various forms and formats. In the example depicted on the right, the document is a screen shot image. More classical document images can be found here.

The interface related to interacting with individual document is made out of three main parts: a top banner, the display area in the middle, and the comment section at the bottom.

  1. The Banner         Banner

    The icons in the top banner provide a set of possible interactions with the document on display. They consist of:

    • downloading the image;
    • copying the image to a private storage area (obsoleted);
    • information related to algorithm interactions with this image (unimplemented);
    • tagging;
    • information related to the dataset hierarchy this document belongs to;
    • displaying and interacting with all annotation areas defined for this document;
    • creating new annotation areas for this document.

    While most of the interaction possibilities are self explaining, the last two, related to interacting with and creating interaction areas, will be explained in more detail in the following section.

  2. The Display Area

    The central display area is of no major interest other than to display the document currently being accessed. It simply shows a thumbnail version of the real document and allows (when the annotation display or creation tools have been selected) to overlay annotation areas over the image. These overlays are clickable, and their use and roles will be detailed further, here.

  3. The Comment Section

    The lower part of the window contains recorded comments, ratings and offers the possibility to the user to add free form comments to the document.

    Comment Section

Annotations

DAE's main goal is not to offer just a nice browsable repository for document images. The interface described in the previous sections is merely a side effect to its underlying storage structure. One of the more essential roles for the platform is to manage the association between documents and the results of document analysis algorithms. One of its effects on the browsing interface is the ability to visualize annotations and interpretations on document images, along with their user-define types, as well as to create new annotations and interpretations.

As mentioned before, Looking glassdocument annotations can be viewed and accessed using the looking glass;Pencil document annotations can be created using the pencil, from the top banner when viewing a specific document.

Note:

Although this part of the description focuses on human interaction based annotation creation, it needs to be stressed that annotations and interpretations can also be automatically created by algorithms or bulk uploaded to the platform.

The manual annotation features are merely one particular aspect of the overall platform.

Underlying Data Model

In order to fully understand the information displayed when requesting annotations and clicking on the looking glass icon, it is important to take a quick look at the underlying DAE data model. The Entity-Relationship Model referred to by the previous link may look impressive to a layman, but there is no need to go through all the details to understand what follows.

The most important items stored in the DAE database are:

  • page_images (document images)
  • page_elements (annotation areas contained within page_images)
    page_elements can be either rectangular bounding boxes or pixmaps.
  • page_element_property_values (or interpretations associated to page_elements)

All of them can have one or more user-defined data types associated to them.

Browsing Existing Annotations and their Interpretations

When clicked, the looking glass will display the list of page_elements defined for the current page_image, on the right hand side of the display pane, and will, simultaneously, draw overlay areas within the display zone, corresponding to these zones. When hovering the mouse pointer over items in the list will highlight the corresponding overlay area. Clicking on either one will open a specific annotation page.

Elements are listed by their primary type and then by type of their corresponding interpretations.Annotation Example

Example:

Select this image and click on the looking glass. The display should look as the one on the right hand side.

There are 2 classes of annotation areas:

  • One labeled "CVC textline" (containing one area called "Text 1")
  • One labeled "Manual Annotation Area" (containing two areas called "Text 1" and "gedi_type_DLLogo 1"

Clicking on the greyed class labels will collapse the menu and remove the corresponding overlays on the image. Re-clicking will make them appear again.

Clicking on either one of the list items or on their corresponding overlay areas will open the corresponding interpretation page.

Interpretations are displayed on a separate page consising of a cropped version of the corresponding annotation area, containing a set of pull-down menus underneath, corresponding to the associated interpretations. The image below provides an example of the text interpretation associated to the "Manual Annotation Area - Text 1" from the previous document.

Interpretation Example

Creating Annotations

The current web interface only allows for rectangular annotation areas, although the underlying data model allows for more elaborate areas, like pixmaps.

To add an annotation area, select the pencil icon, draw the area, and associate a type. The type can either be a pre-existing type or a new user defined string. The interface will present the possible pre-existing types by offering those already present within the document first.

Creating Interpretations

Creating interpretations is straightforward. Once on the interpretation page of a page_element (e.g. here) just select "Add Interpretation" at the bottom of the display area and follow the instructions there. The two steps to adding an interpretation is to associate a type and a value to the selected element. The type can either be a pre-existing type or a new user defined string. The value of the interpretation can be anything.