Quantcast
Channel: Recent Questions - Stack Overflow
Viewing all articles
Browse latest Browse all 12111

parse csv data with databricks

$
0
0

There is a csv file, content like( splitter is \u0001)enter image description heretry read as a table :

SELECT * FROM read_files('{csv_file_path}',  format => 'csv',  header => false,  sep=>"\u0001",  quote=>'"',  escape=>'\\',   charToEscapeQuoteEscaping=>'\0',   nullValue=>'\\N',  schema => 'id int, name string, ts string')

get below result:enter image description here

the record id=3 is parse failed.here is expect:

1 "abc    null2 \abc    null3 ab"""c\ null4 abc"    null5 \"abc   null6 ab\\c   null7 \\"abc  null8 abc\\"  null

Can anybody help here on how to parse the csv file? Thanks a lot. I tried a lot of csv options, and did not work.


Viewing all articles
Browse latest Browse all 12111

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>