NiagaraST

The Niagara project is a joint effort between the University of Wisconsin-Madison and Portland State University to study mechanisms for effectively querying the Internet. At PSU, we are studying queries over data streams and XML data streams in particular. We have developed NiagaraST, a data stream management system based on the Niagara 1.0 code base from the University of Wisconsin-Madison. NiagaraST uses a novel approach for processing window queries that significantly reduces the amount of buffering required for such queries and which allows processing of disordered data. In addition, we have developed NEXMark, a benchmark for data stream queries and implemented XMLB, a fantasy baseball demo, showcasing our ideas for stream processing.

Research

• Mechanisms for accumulation and aggregation of XML data.

• Punctuation semantics for querying continuous data streams.

• Partial evaluation of distributed queries with mutant query plans.

• Evaluation of window aggregates over data streams.

• Inter-Operator Feedback via Punctuation to support various processing optimization schemes

People

• Prof. David Maier

• Prof. Kristin Tufte

• Amit Bhat

• Rafael J. Fernández-Moctezuma

• James Whiteneck

• Peter Tucker (now at Whitworth College)

• Jin Li (now at Amazon.com)

• Vassilis Papadimos (now at Microsoft)

Publications

• Rafael J. Fernández-Moctezuma, Kristin A. Tufte, and Jin Li. Inter-Operator Feedback in Data Stream Management Systems via Punctuation. CIDR 2009.

• Li, J., Tufte, K., Shkapenyuk, V., Papadimos, V., Johnson, T., Maier, D. Out-of-Order Processing: A New Architecture for High-Performance Stream Systems, VLDB 2008

• Kristin Tufte, Jin Li, David Maier, Vassilis Papadimos, Robert L. Bertini, and James Rucker Travel time estimation using NiagaraST and latte Demonstration: 2007 ACM SIGMOD Conference on Management of Data, Beijing, China, June 2007.

• Tucker, P., Maier, D., Sheard, T., Stephens, P. Using Punctuation Schemes to Characterize Strategizes for Querying Data Streams IEEE Transactions on Knowledge and Data Engineering Volume 19, 2007.

• Jin Li, David Maier, Kristin Tufte, Vassilis Papadimos, and Peter A. Tucker. Semantics and Evaluation Techniques for Window Aggregates in Data Streams. In Proceedings of the 2005 ACM SIGMOD Conference on Managment of Data, Baltimore, MD, June 2005.

• David Maier, Jin Li, Peter Tucker, Kristin Tufte, and Vassilis Papadimos. Semantics of Data Streams and Operators. ICDT 2005 (Invited Presentation). Slides.

• Jin Li, David Maier, Kristin Tufte, Vassilis Papadimos, and Peter A. Tucker. No Pane, No Gain: Efficient Evaluation of Sliding-Window Aggregates over Data Streams. SIGMOD Record, March 2005.

• Vassilis Papadimos, David Maier, and Kristin Tufte. Distributed Query Processing and Catalogs for Peer-to-Peer Systems. CIDR 2003.

• Vassilis Papadimos and David Maier. Distributed Queries without Distributed State. WebDB 2002.

Source Code

Funding

Funding for this work is provided by NSF ITR award IIS0086002, and by DARPA through NAVY/SPAWAR contract no. N66001-99-1-8908.