SoBigData · Project

Open Big Data Platform That Turns Social and Digital Traces Into Actionable Insights

digitalTestedTRL 5Thin data (2/5)

Imagine every click, tweet, and online purchase leaves a tiny digital footprint. Now imagine a giant shared workshop where data scientists across Europe can bring those footprints together, combine tools, and figure out what people actually do — not what surveys say they do. SoBigData built exactly that: a shared online platform with ready-to-use data mining tools, open datasets, and virtual labs. Think of it as a communal kitchen for data cooks, where everyone shares recipes and ingredients instead of working alone in separate kitchens.

By the numbers

consortium partners

countries represented

5,000,000

EUR EU contribution

total project deliverables

industry partner ratio

The business problem

What needed solving

Companies today sit on mountains of digital customer data — clicks, purchases, social media interactions — but lack the specialized tools and methods to extract meaningful behavioral patterns. Building these analytical capabilities in-house requires hiring scarce data scientists and developing custom software, which is expensive and slow. Meanwhile, competitors who crack the code on understanding digital behavior gain a decisive market advantage.

The solution

What was built

The project built a working e-infrastructure platform with virtual research environments, two versions of an Innovation Accelerator web platform with impact metrics, and multiple software releases containing social data mining tools. In total, 57 deliverables were produced, including open datasets and open-source analytical methods for mining social and behavioral data.

Audience

Who needs this

Data analytics consultancies looking for ready-made social mining toolkitsE-commerce companies analyzing customer behavior patterns at scaleDigital marketing agencies tracking content spread and audience engagementCity governments or smart-city consultancies using mobility and social dataMarket research firms replacing surveys with behavioral data analysis

Business applications

Who can put this to work

Retail and E-commerce

mid-size

Target: Online retailers or consumer brands analyzing customer behavior

If you are an e-commerce company dealing with understanding why customers abandon carts or how social trends drive purchases — this project developed an open e-infrastructure with social data mining tools that can analyze digital behavioral traces at scale. With 57 deliverables including multiple software releases and a virtual research environment, the platform provides ready-made analytical methods you can adapt rather than building from scratch.

Media and Advertising

any

Target: Digital marketing agencies or media companies tracking audience engagement

If you are a media company dealing with understanding audience behavior across platforms — this project built an Innovation Accelerator web platform that delivers impact and innovation metrics for publications and collaborations. The ecosystem connects datasets from 12 partner institutions across 7 countries, giving you access to social mining methods that reveal how content spreads and influences behavior.

Public Policy and Urban Planning

enterprise

Target: City governments or consultancies advising municipalities on data-driven decisions

If you are a city administration or consultancy dealing with understanding mobility patterns, social dynamics, or citizen behavior — this project created a pan-European research infrastructure with open-source methods and openly available datasets specifically designed for social data mining. The platform was built by 10 universities and 2 research organizations, ensuring scientific rigor behind the analytical tools.

Frequently asked

Quick answers

What would it cost to access or use this platform?

SoBigData was funded with EUR 5,000,000 in EU contribution and designed as an open research infrastructure. The methods and datasets are openly available, meaning access to the core tools is free. However, customization, integration, or dedicated support would likely require a service agreement with one of the 12 consortium partners.

Can this handle industrial-scale data volumes?

The platform was designed as a large-scale research infrastructure handling big data from social and digital sources across Europe. Multiple software releases and a dedicated e-infrastructure with virtual research environments were delivered, indicating capacity for substantial data volumes. Exact throughput benchmarks are not specified in available project data.

What about intellectual property and licensing?

Based on available project data, the infrastructure promotes open science with openly available datasets and open-source methods. This suggests permissive licensing for the core tools. Specific licensing terms for individual components should be confirmed with the coordinator, Consiglio Nazionale delle Ricerche in Italy.

Is the platform still operational after the project ended in 2019?

The project closed in December 2019, but it was designed to establish a lasting research infrastructure. A successor project (SoBigData++) continued the work. The platform at sobigdata.eu may still be accessible, but current operational status should be verified directly.

How does this integrate with our existing data systems?

The e-infrastructure was built with virtual research environments (VREs) that allow integration of external datasets and methods. The Innovation Accelerator platform was customized through two iterations of software prototypes. Based on available project data, the architecture supports reuse and comparison of data and services, suggesting API-level integration is feasible.

Is this compliant with GDPR and data ethics requirements?

The project explicitly emphasizes ethic-sensitive scientific discoveries and was built with ethical considerations as a core design principle. Given the consortium of 10 universities and 2 research organizations across 7 EU countries, GDPR compliance was a requirement. Specific compliance certifications should be confirmed with the consortium.

What kind of support is available?

The consortium includes 12 partners across 7 countries with extensive training and networking activities built into the project. Based on available project data, the infrastructure was designed to foster a community of social data scientists, meaning documentation and community support channels exist. Commercial support arrangements would need to be negotiated with individual partners.

Consortium

Who built it

This is a purely academic consortium — 10 universities and 2 research organizations across 7 countries (Italy, Germany, Finland, Netherlands, UK, Estonia, Switzerland), with zero industry partners and zero SMEs. The coordinator is Italy's National Research Council (CNR), one of Europe's largest public research bodies. For a business looking to adopt this technology, the absence of commercial partners means there is no built-in path to market. You would be working directly with academic institutions, which typically means slower response times but deep technical expertise. The EUR 5,000,000 budget and 57 deliverables indicate substantial development effort, but commercialization would require a dedicated transfer partner.

CONSIGLIO NAZIONALE DELLE RICERCHECoordinator · IT
TARTU ULIKOOLparticipant · EE
THE UNIVERSITY OF SHEFFIELDparticipant · UK
AALTO KORKEAKOULUSAATIO SRparticipant · FI
UNIVERSITA DI PISAparticipant · IT
SCUOLA NORMALE SUPERIOREparticipant · IT
EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICHparticipant · CH
SCUOLA IMT (ISTITUZIONI, MERCATI, TECNOLOGIE) ALTI STUDI DI LUCCAparticipant · IT
TECHNISCHE UNIVERSITEIT DELFTparticipant · NL
KING'S COLLEGE LONDONparticipant · UK
GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET HANNOVERparticipant · DE

How to reach the team

Consiglio Nazionale delle Ricerche (CNR), Italy — contact through their technology transfer office

Order a report on this consortium See CONSIGLIO NAZIONALE DELLE RICERCHE profile →

Next steps

Talk to the team behind this work.

Want to explore how SoBigData's social mining tools could work for your business? SciTransfer can arrange a focused introduction with the right technical team at CNR.

Order a report on this topic More in Digital & ICT