Child tables that are elevated to the parent as a concatenated string are correctly quoted in the generated CSV as part of DuckDBDialect.insert_into_temp_tables. However, when the file is then read for insert, the quote char is auto detected. If no rows are quoted in the sample used for the auto detect, then the correct quote char is not detected. This in turn causes the insert to fail as the incorrect number of columns are detected.
This can be resolved by specifying the quote char in the SQL statement:
sql = text( f"INSERT INTO {full_name} ({insert_cols}) " f"SELECT {select_exprs} " f"FROM read_csv('{safe_path}', header=true, nullstr='', all_varchar=true, quote='\"')" )
Child tables that are elevated to the parent as a concatenated string are correctly quoted in the generated CSV as part of DuckDBDialect.insert_into_temp_tables. However, when the file is then read for insert, the quote char is auto detected. If no rows are quoted in the sample used for the auto detect, then the correct quote char is not detected. This in turn causes the insert to fail as the incorrect number of columns are detected.
This can be resolved by specifying the quote char in the SQL statement:
sql = text( f"INSERT INTO {full_name} ({insert_cols}) " f"SELECT {select_exprs} " f"FROM read_csv('{safe_path}', header=true, nullstr='', all_varchar=true, quote='\"')" )