User:Travlr/ProDataLab

ProDataLab is an flow-based programming (FBP) environment that takes advantage of every possible efficiency in processing data via streams. The intention is to build an extremely flexible, parallel, infinitely scalable and distributed application-services system for every type and size of data set — processing, analyzing, learning, plotting and reporting, etc. Fast, efficient ZeroMQ messaging will be used as the core of ProDataLab's communication fabric. ProDataLab will eat extremely large data sets for breakfast.

The following features are part of the road map:

Everything comprised in the software system is a loosely coupled component.. user interface, scheduler-router, as well as each application service.
Inspired by, and to be modeled upon flow-based programming (FBP) originating from J. Paul Morrison.
The first user interface to be designed will be a visual programming environment for the personal computer and written in Qt with platform portability in mind.
Automatic recognition and adaption of all available hardware resources will be a strong focus including single-core, multi-core, networked remotes, GPU, as well as antiquated processing hardware.
Automatic recognition and adaption of processor-network throughput and latencies for most efficient dispatching, scheduling and routing.
Utilization of all available resources for automated concurrency and massive parallelism.
Ability for community based, distributed processing and data sharing similar to a decentralized seti@home and folding@home.
Component application service types, can and will include anything related to data and will by no means be limited to: statistics, visualization, financial markets, database access, plotting, machine learning, business enterprise solutions, imaging, medical, etc.
Thanks to ZeroMQ, software components can be contributed to the system easily from approximately 20 different programming languages, and growing.