This repository contains several scripts to verify the complete DataVaccinaor Vault API functionality. It also offers single and multiple instances performance measurement.
The protocol test script needs the url (and port) of the service to test. Call it like this:
php vaccinator_test.php http://127.0.0.1:8080
The above will connect to localhost on port 8080.
php vaccinator_test.php https://service.mydomain.com
The above will connect using https connection to service.mydomain.com (port 443 by default).
All tests have to return like this (positive example result):
Get version and check availability: - pass NOTE: Server supports 'search' module. We will test this, too. NOTE: Server supports 'publish' module. We will test this, too. Testing invalid requests (wrong data, missing data etc): - pass - pass - pass - pass - pass Testing to add data: NOTE: New user VID: 8111823e306f95b93f67559aa205b874 - pass Testing to publish data: - pass - pass NOTE: New published user VID: 22eb8014df566c4ccceb2c0ec8109785 - pass Tests updating data: - pass - pass - pass - pass Tests retrieving data: - pass - pass Testing 'search' plugin functions: - pass - pass Testing 'publish' plugin functions: - pass - pass Cleanup vid's created: Removing vid [8111823e306f95b93f67559aa205b874]... OK Removing vid [22eb8014df566c4ccceb2c0ec8109785]... OK Done
This is an example call to measure the localhost system performance:
php perf_test.php 1 vaccinator http://localhost:8080 30 4
The above call will do a test with:
-
provider id 1
-
provider password vaccinator
-
system at localhost port 8080
-
running for 30 seconds
-
utilizing 4 instances
Output:
Start client 0... Start client 1... Start client 2... Start client 3... Connection to http://localhost:8080 is working. Duration: 31 sec Transactions: 209 (each create, retrieve and delete) Operations/sec: 20.225806451613 Connection to http://localhost:8080 is working. Duration: 31 sec Transactions: 211 (each create, retrieve and delete) Operations/sec: 20.41935483871 Connection to http://localhost:8080 is working. Duration: 31 sec Transactions: 224 (each create, retrieve and delete) Operations/sec: 21.677419354839 Connection to http://localhost:8080 is working. Duration: 31 sec Transactions: 223 (each create, retrieve and delete) Operations/sec: 21.58064516129 Done
All the instances return their own metrics. To get the final metrics, you can add the values. In the above example, each instance did about 20 operations/sec. So the overall number of operations was 4 x ~20 = ~80 operations/sec. Doing the exact calculation, it was 83.91 operations/sec or 302K operations/hour.
|
ℹ️
|
The above example was generated on a virtual machine (VBox): Operating System: KDE neon 5.19 This virtual machine was also running the connected single instance of Cockroach Database and the test script itself. ~50% of the system performance was utilized by Cockroach DB, ~20% by dv-vault executable. Please note that a production system with 4 CPU cores and 16GB of memory would do much more! For example, on a 2vCPU/4GB RAM system at a German hoster, we got up to 330 operations/sec (using 20 threads while the CockroachDB had two nodes, one local and one in Helsinki with 25ms latency). |
You can use a placeholder with the following scheme to generate random numbers in your URL:
[min-max]
This will get replaced by a number between given min and max.
Example:
php perf_test.php 1 vaccinator https://dc[1-3].sirius.datavaccinator.com 10 4
The above will generate URLs with a random number between 1 and 3 like for dc1.sirius.datavaccinator.com or dc2.sirius.datavaccinator.com.
This allows you to test access to multiple datavaccinator instances, all connected to the same database cluster.