We research methods and tools that allow users of all technical levels to clean, manipulate and wrangle their datasets painlessly. Check out our latest publications!

News & Updates

Projects

D4

Data-driven domain discovery (D4) aims to identify sets of terms that represent instances of a semantic concept (domain) in a given collection of structured (tabular) datasets. D4 gathers contextual information for terms based on the columns they occur in and the terms the co-occur with. Context information is used to cluster terms with similar contexts into semantic domains.

Vizier

Vizier features an intuitive interface combining elements of notebooks and spreadsheets, allowing analysts to quickly see, edit, and revise data. This capability is complemented by a framework for automated data cleaning steps that are seamlessly integrated with manual curation operations. The heart of Vizier is a system that tracks the provenance of curation workflows and data.

Profiling

Our profiling work ranges from type detection, geospatial and temporal profiling to regex pattern detection in text. They have been developed as stand alone libraries that can easily be integrated into any project and used readily.

Openclean is a Python software library that provides access to algorithms for a wide range of data curation tasks, including data standardization, constraint repair, and data profiling. Openclean is easy to extend with new functionality and defines a simple API to combine operators into complex data curation pipelines.

Publications

Data-Driven Domain Discovery for Structured Datasets.

Masayo Ota, Heiko Mueller, Juliana Freire, Divesh Srivastava.

Proceedings of the VLDB Endowment, 13(7), 2020.

Your notebook is not crumby enough, REPLace it.

Mike Brachmann, William Spoth, Oliver Kennedy, Boris Glavic, Heiko Mueller, Sonia Castelo, Carlos Bautista, Juliana Freire.

Conference on Innovative Data Systems Research (CIDR), 2020.

Data Debugging and Exploration with Vizier.

Mike Brachmann, Carlos Bautista, Sonia Castelo, Su Feng, Juliana Freire, Boris Glavic, Oliver Kennedy, Heiko Mueller, Remi Rampin, William Spoth, Ying Yang.

ACM International Conference on Management of Data (SIGMOD), Demo Track, 2019.

Data Quality: The Role of Empiricism.

Shazia Wasim Sadiq, Tamraparni Dasu, Xin Luna Dong, Juliana Freire, Ihab F. Ilyas, Sebastian Link, Renée J. Miller, Felix Naumann, Xiaofang Zhou, Divesh Srivastava.

SIGMOD Rec. 46(4): 35-43, 2017.

The exception that improves the rule.

Juliana Freire, Boris Glavic, Oliver Kennedy, Heiko Mueller.

Workshop on Human-In-the-Loop Data Analytics (HILDA), 2016.

Exploring What not to Clean in Urban Data: A Study Using New York City Taxi Trips.

Juliana Freire, Aline Bessa, Fernando Chirigati, Huy T. Vo, Kai Zhao.

IEEE Data Eng. Bull. 39(2): 63-77, 2016.

RioBusData: Outlier Detection in Bus Routes of Rio de Janeiro.

Aline Bessa, Fernando de Mesentier Silva, Rodrigo Frassetto Nogueira, Enrico Bertini, Juliana Freire.

Symposium on Visualization in Data Science (VDS at IEEE VIS), 2015.