SPARKSQL RDF Benchmark.
A systematic Benchmarking on the performance of Spark-SQL for processing Vast RDF datasets
- Home
- Bench-Ranking
- Ranking Criteria
- Ranking Goodness
- Project Source Code
- Results
- Benchmark Datasets
- Download Logs
This project is maintained by DataSystemsGroupUT
Distributed Experiments
Hardware and Software Configurations: Our experiments have been executed on a bare metal cluster of four machines with a CentOS-Linux V7 OS, running on a 32-AMD cores per node processors, and 128 GB of memory per node, alongside with a high speed 2 TB SSD drive as the data drive on each node. We used Spark V2.4 to fully support Spark-SQL capabilities. We used Hive V3.2.1. In particular, our Spark cluster is consisted of one master node and three worker machines, while Yarn is used as the resource manager, which in total uses 330 GB and 84 virtual processing cores.
- In the following figures we cateogrized running times queries into Long-Running Queries (Q1,Q3, Q10, Q11), and Short-Running Queries. We think that this grouping makes the queris plots looks clearer than the grouped query runtimes plots, due to they have big different running times.
Execution Runtimes (100M Triples Dataset Results)
100M Triples Dataset Results Figures
- [Horizontally Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Predicate-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Subject-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
250M Triples Dataset Results Figures
- [Horizontally Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Predicate-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Subject-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
500M Triples Dataset Results Figures
- [Horizontally Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Predicate-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
- [Subject-Based Partitioned]
- Single Table Schema (ST) Short(left) and Long (right) Queries:
- Vertical Tables Schema (VT), Short(left) and Long (right) Queries:
- Property Tables Schema (PT), Short(left) and Long (right) Queries:
