Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error running data fusion queries - Physical input schema should be the same as the one converted from logical input schema #655

Closed
a-agmon opened this issue Oct 2, 2024 · 2 comments · Fixed by #664

Comments

@a-agmon
Copy link
Contributor

a-agmon commented Oct 2, 2024

I have a simple iceberg table that I can query using DataFusion

+----+----------+-----+------------+
| id | name     | age | dt         |
+----+----------+-----+------------+
| 2  | Jane     | 20  | 2024-01-02 |
| 1  | John     | 10  | 2021-01-02 |
...

However, whenever I run a query such as select count(*) from tbl or select name from tbl group by 1 I get the following exception:

Error: Internal error: Physical input schema should be the same as the one converted from logical input schema..

If this is reproducible with others, then I think we have some issue with the provider.

The code I use to run queries against the table:

let warehouse_location = String::from("/tmp/testwarehouse");
let file_io = FileIO::from_path(warehouse_location)?.build()?;
let metatdata_location = "/tmp/testwarehouse/test/test_table/metadata/v1.metadata.json";
let table_indent = TableIdent::from_strs(["test", "test_table"])?;
let static_table =
    StaticTable::from_metadata_file(metatdata_location, table_indent, file_io).await?;
let table = static_table.into_table();
let ctx = SessionContext::new();
let table_provider = IcebergTableProvider::try_new_from_table(table).await?;
ctx.register_table("tbl", Arc::new(table_provider))?;
let df = ctx.sql("select count(*) from tbl").await?;
df.show().await?;
@FANNG1
Copy link
Contributor

FANNG1 commented Oct 10, 2024

seems caused by apache/datafusion#12687, I will propose a PR to upgrade datafusion version.

@a-agmon
Copy link
Contributor Author

a-agmon commented Oct 10, 2024

Thanks @FANNG1
FYI @sdd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants