Demonstration of a real-time process baked by ViyaDB.
The process uses TestContainers framework for starting, and orchestrating needed Docker containers.
+--------------------+ +-----------+ +-----------------+
| | | | | |
| | pipe | | | |
| Events Generator |--------> | Kafka | | Configuration |
| (Docker) | | (Docker) | | (Consul Docker) |
| | | | | |
+--------------------+ +-----------+ +-----------------+
| ^
| |
v |
+--------------------+ |
| | |
| | |
| Spark Streaming |------------+
| |
| |
+--------------------+
|
| notify on new micro-batches
v
+--------------+
| |
| Kafka |
| |
+--------------+
|
| listen to micro-batches, and load them
v
+----------------+
| |
| ViyaDB |
| (Docker) |
| |
+----------------+
- Leiningen >= 2.7.1
lein uberjar
First, build, and put viyadb-spark*.jar file into current directory. Then, run the following command:
java -jar target/pipeline-demo-0.1.0-standalone.jar
When running for the first time, it might take some time to pull all required Docker images, please be patient.