'Azure Data Factory 2, Data flow with derived column, how to set a timestamp to null
I have a Azure DataFactory V2 pipeline that executes a data flow.
The Azure Sql source table have a Date not null column, DateReported. This is imported from an external source. Where null values are 1899-01-01.
The destination Sql table have the same field, DateReported but as Date null
I'm using a Derived Column in my dataflow to sanitize data. I want to insert null when the date is 1899-01-01
My derived column function is this
iif(!(year(DateReported) == 1899 || year(DateReported) == 1753)
, DateReported
, null()
)
This give me the error "Expression should return the same type 'timestamp' as previous expressions".
If I invert the iif to this
iif((year(DateReported) == 1899 || year(DateReported) == 1753)
, null()
, DateReported
)
I get the error "Expression should return the same type 'null' as previous expressions"
I can fix this in sql from the source like this
Select ...
DateReported2 =
CASE
WHEN DateReported is null THEN DateReported
WHEN YEAR(DateReported) = 1899 THEN NULL
ELSE DateReported
End
...
But this is messy, as all my other logic are in the dataflow.
How can I create a derived column that is a nullable timestamp, like C# DateTime? or SSIS NULL(DT_DATE)?
Solution:
case(year(DateReported) != 1899, DateReported)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|