Granules: Distributed Real time Stream Processing
Sponsor:
	US National Science Foundation Department of Homeland Security

Granules supports the processing of data streams over a distributed collection of processing elements. Such streams can be generated in settings involving observational and monitoring equipment, simulations, and computational workflows. In Granules these computations can be long running, with multiple rounds of execution, with the ability to retain state across successive rounds. Granules allows a collection of related computations to be expressed as directed graphs that have cycles in them, and orchestrates the completion of such distributed processing. Granules manages the lifecycle and finite state machine associated with computations. The system can orchestrate such stream processing computations within traditional clusters, collection of desktops, or IaaS VM-based settings. The processing encapsulated within these computations can be arbitrary, and encoded in C, C++, C#, Java, R and Python. Granules also incorporates support for variants of the MapReduce paradigm that make it amenable for scientific applications. By abstracting the complexities of doing I/O and the vagaries of execution in distributed settings, Granules allows a domain scientist to focus on the problem on at hand and not on the artifacts related to deployments in large-scale distributed systems. A broad class of compute and data intensive applications can benefit from the capabilities available in Granules. Some of the application domains that Granules is currently deployed in include brain computer interfaces, epidemiological modeling, handwriting recognition, data clustering algorithms, and bio-informatics (mRNA sequencing).

NaradaBrokering: Dissemination of voluminous data streams
Sponsors:
	US National Science Foundation Open Middleware Infrastructure Institute, UK US Department of Energy

NaradaBrokering is an infrastructure for building large-scale distributed systems that are secure and failure-resilient. The NaradaBrokering substrate itself comprises a distributed network of cooperating broker nodes. The substrate places no constraints on the size, rate and scope of the interactions encapsulated within the streams, or on the number of entities within the system. It also incorporates several services to mitigate network-induced problems as streams traverse disparate domains during traversals. The system provisions these guarantees such that they are easy to harness, while delivering consistent and predictable performance that is adequate for use in real-time settings.

Research in NaradaBrokering has been funded through grants from funding agencies in the United States and the United Kingdom. The project is funded through two grants from the National Science Foundation, two grants from the Open Middleware Infrastructure Institute of the United Kingdom, and an STTR grant from the Department of Energy