Skip to content

fix: override is_string/is_number/is_float type classification for Databricks types#1390

Open
psaikaushik wants to merge 1 commit intodatabricks:mainfrom
psaikaushik:fix/1380-column-type-classification
Open

fix: override is_string/is_number/is_float type classification for Databricks types#1390
psaikaushik wants to merge 1 commit intodatabricks:mainfrom
psaikaushik:fix/1380-column-type-classification

Conversation

@psaikaushik
Copy link
Copy Markdown

Summary

col.is_string(), col.is_number(), col.is_float(), and col.is_numeric() all return False for Databricks/Spark column types because the base Column class only recognizes PostgreSQL-style type names.

Closes #1380
Related: dbt-labs/dbt-spark#658

Root Cause

The base Column class from dbt-adapters defines is_string() etc. using type name sets that include PostgreSQL types (text, character varying, etc.) but not Spark/Databricks types (string, int, bigint, double, etc.).

DatabricksColumn inherits from SparkColumn which inherits from the base Column, but neither overrides these classification methods.

Fix

Override is_string(), is_number(), is_float(), is_integer(), and is_numeric() on DatabricksColumn with the complete set of Spark/Databricks type names:

  • String types: string, varchar, char, text, etc.
  • Numeric types: tinyint, smallint, int, integer, bigint, long, float, double, decimal, numeric, real (plus decimal(p,s) variants)
  • Float types: float, double, real
  • Integer types: tinyint, smallint, int, integer, bigint, long

Verification

Column: name03 | Data Type: string | Is String: True  (was: False)
Column: age    | Data Type: int    | Is Number: True  (was: False)

Checklist

  • No breaking changes — only adds overrides, doesn't change any existing behavior for correctly-typed columns
  • Covers all Spark/Databricks primitive types

…lumn

The base Column class type classification methods don't recognize
Spark/Databricks type names like 'string', 'int', 'bigint', 'double'.
This causes col.is_string() to return False for string columns and
col.is_number() to return False for numeric columns.

Override these methods on DatabricksColumn with the complete set of
Databricks/Spark type names.

Closes databricks#1380
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt Classes Instance Methods is_string, is_numeric, is_number, is_float don't seem to work

1 participant