Skip to content

Commit

Permalink
Minor: improve some docstrings about pruning (apache#3041)
Browse files Browse the repository at this point in the history
  • Loading branch information
alamb authored Aug 6, 2022
1 parent 941b9a2 commit 0b3130b
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 8 deletions.
12 changes: 6 additions & 6 deletions datafusion/core/src/physical_optimizer/pruning.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@
//! significant performance improvements by avoiding the need
//! to evaluate a plan on entire containers (e.g. an entire file)
//!
//! For example, it is used to prune (skip) row groups while reading
//! parquet files if it can be determined from the predicate that
//! nothing in the row group can match.
//! For example, DataFusion uses this code to prune (skip) row groups
//! while reading parquet files if it can be determined from the
//! predicate that nothing in the row group can match.
//!
//! This code is currently specific to Parquet, but soon (TM), via
//! <https://github.com/apache/arrow-datafusion/issues/363> it will
//! be genericized.
//! This code can also be used by other systems to prune other
//! entities (e.g. entire files) if the statistics are known via some
//! other source (e.g. a catalog)

use std::convert::TryFrom;
use std::{collections::HashSet, sync::Arc};
Expand Down
5 changes: 3 additions & 2 deletions datafusion/core/tests/parquet_pruning.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// This file contains an end to end test of parquet pruning. It writes
// data into a parquet file and then
//! This file contains an end to end test of parquet pruning. It writes
//! data into a parquet file and then verifies row groups are pruned as
//! expected.
use std::sync::Arc;

use arrow::array::Decimal128Array;
Expand Down

0 comments on commit 0b3130b

Please sign in to comment.