Running VLDB Paper experiments

[nitinm25]

Issue Type

Perfomance

Have you searched for existing issues?

Yes

Link to Relevant Documentation

https://github.com/secretflow/scql/blob/vldb2024/vldb/README.md

Question Details

Hi,

I have been running the experiments referenced in the VLDB paper "SecretFlow-SCQL: A Secure Collaborative Query Platform" using the `vldb2024` branch.

I've modified the scripts slightly so I can run them in a 2-machine LAN setup.
I am currently using an image built on the `vldb2024` branch (`scql:vldb`)

I had a few questions regarding these experiments:
1.) Is there a way to find out how many tuples are processed under MPC vs Plaintext for a given query?
2.) Can the Sorting experiments be run in a LAN setup? I noticed the branch's experiment simulates WAN using the tc command on the loopback interface.

Thank you for your help!

[jingshi-ant]

1.）all tuples are stored in TensorTable(private)/SymbolTable(secret), you can print out the tuple shapes in GetTensor/AddTensor to show tuples processed under Plaintext, and for MPC, you may need to add logs in SPU or print shapes in every related operators(e.g add 'SPDLOG_INFO("size {}, ", value.numel()); to line73 and line 81 to operator/sort.cc)

[jingshi-ant]

2.) for sort exp to run in 2-machine, you may need to modify codes in repo secretflow/spu, you can write binary to run the radixsort only.
by the way, the scql group_by test also contains the time cost of sort api in spu(you need to print it out), which may be helpful