“deecoob insight” – Smart data platform for various applications

deecoob insight automatically collects mass data from different public and private sources for you. The software analyzes collected data to clean, filter, cluster and rank it and to enrich and complete it with missing data. The quality and quantity of the data as well as the performance of automatic processing are continually measured and evaluated by deecoob insight. Key performance indicators (KPI) are defined to achieve objective facts and figures, graphically displayed in real time using dashboards.

deecoob insight currently stores and processes over 30 million records. Sources such as Facebook, epaper and web pages are continuously scanned. Over 200,000 new records are collected and evaluated every week. More than 30 employees qualify over 10,000 records manually every day and hereby reducing them to several hundred economically relevant records for our customers.

Web crawler

The web crawler scans any web page for data and information that is relevant for our customers. Depending on the specific demand and necessity, the crawler scans complete websites of cultural institutions, hotels, restaurants, authorities, companies and many more. The websites, as source of desired data, can be classically unstructured (e.g. home or landing page) or structured (e.g. online catalogs, online shops, event calendars, etc.). The crawler only needs the internet addresses (URLs) as entry points. You can provide us with URLs from your CRM- or ERP-Software. The crawler can also “get” the URLs independently on the basis of predefined criteria from online directories, google places, open-streetmap, domain directories or other digital sources.

The crawler takes one URL at a time and searches the entire web page of the respective URL for predefined data, information, words, phrases, or other properties relevant to the particular customer. In doing so, the crawler can search as many web addresses as desired successively. The crawler can also search the same URLs in intervals for new, changed or additional information. When the crawler finds relevant information on a web page, this data can immediately be matched with reference data in order to decide whether the data must be stored locally. If this is the case, the crawler automatically transfers the information to a local data storage. In addition, the crawler can also create and save a pdf document or screenshot from the web page as evidence.

After crawling and storing, the software provides the data to further text and data mining, qualification and modelling according to customer demand (see functions).

Social network crawler

The social network crawler works similarly to the web crawler. By means of defined “entry points” (e.g. Facebook, Twitter, Xing, Linkedln, Google+) and criteria, it scans the digital source for relevant information. For Facebook, one can differentiate between information which is found on Facebook profiles or on the Facebook timeline or in both. This way, deecoob insight “collects” millions of social network records every day, every week, every month, and puts them into further processing.

After crawling and storing, the software provides the data to further text and data mining, qualification and modelling according to customer demand (see functions).

ePaper crawler

The core competence of the epaper crawler is the automated analysis and collection of relevant data from electronic newspapers. These can be daily, weekly or monthly newspapers free of charge or subscription newspapers. The decisive factor is that they are provided by the publisher in digital form (e.g. pdf) via the internet. In the case of subscribed newspapers, we check whether we already have the subscription on deecoob insight or whether this still needs to be deducted.

The newspapers relevant to your data collection are jointly defined and configured by deecoob for automated download. Depending on the appearance interval, the download can be carried out automatically on a daily, weekly or monthly basis. At present, more than 1,000 newspapers of all editions of various years are already available in German on deecoob insight. After having been downloaded, the ePaper is automatically stored and indexed for deecoob insight. Depending on customer requests, it is then forwarded to further text and data mining processes, qualification and modeling for the customer (see functions).

Data processing

The main features of deecoob insight and deecoob service are the combination of an automated procurement and processing of mass data and the manual qualification of this data for specific customer requirements. deecoob insight can perform many processing steps automatically, hereby learning and optimizing itself. However, the results of the software still require extensive testing and qualification by the research professionals of deecoob service or by customer employees. The data processor is deecoob insight´s user interface that provides users with sepecific features and functions:

• Displaying relevant data and records
• Checking for completeness and correctness
• Merging and de-duplication of records
• Verification of assignment (e.g. data record to user location, partner)
• Checking for relevance
• Completion of missing data and information
• Collecting additional records from manual search
• Gathering comments, status, reminder, article numbers
• Searching for records
• Checking evidence, making markings, uploading

You can work with the data processor as a client application using a VPN connection to deecoob insight from anywhere in the world. The user interface is modern, structured, tidy, ergonomic and performant. The functions and features can be adapted and expanded at any time by deecoob for specific customer applications and uses.

Data mapping

Visual representations of data are helpful to quickly discover and structure data, to visualize data cluster or geographical distribution of data. The platform provides data mapping functions for these purposes. With their help, one can, for example, display geographical distribution areas of epaper through polygons on digital maps, in order to recognize places, facilities (e.g. companies) or overlaps in the dissemination area. In addition, geographic allocation allows automated procurement of contact details and from Google Places or Open Street Map.

If you wish to visualize and analyze the geographical distribution of certain elements (eg facilities of an industry, events of a musical type, etc.), you can use deecoob insight´s headmap options. With headmaps it is easy to recognize clustering of data by means of markers, colors or other features. With targeted drilldown, the representation of the mass data (headmap), down to individual data sets, can be gained.

Data discovery

At the beginning of each search, one needs to consider which data is actually needed, where and how it can most effectively be procured and how it can be analyzed and qualified. deecoob insight already provides millions of different records that have been collected and maintained by the platform over many years. Therefore, it is useful to search available data on deecoob insight first. You will find out that many relevant data are already available. This way, effort for addtional implementation of data procurement on the platform can be minimized. Data discovery for concrete searches in our mass data is the basis for:

• Data management – data models that fit the query
• Knowledge discovery – different and flexible views on data
• Statistical analytics – e.g. time- or location-related considerations
• Predictive analytics – trends, best and worst cases scenarios
• Data interaction – zoom in / out, drill down

The information from the text analysis must be visualized according to the application and context-sensitivity. Datahead offers possibilities for visual analytics (e.g. dashboards, directly in applications). With the help of deecoob you are able to interpret data models visually.

Test deecoob insight now.