Quantcast
Channel: Recent Questions - Stack Overflow
Viewing all articles
Browse latest Browse all 12111

Filter a CSV file that has text above column names that must be maintained after the filter process

$
0
0

I work with hundreds of CSV files that have three lines of text above multiple column headers.CSV example.

These files can be as large as 300MB. They need to be filtered down to < 1/3 the size in local preprocessing, before the result is pushed to a server for processing. Filtering is fine when the three lines are removed on upload.

A <- read_csv_arrow( p , as_data_frame = TRUE, skip = 3) 

But the problem is replacing the text (the first three lines) in the same state post filtering, as that information is relevant in a later processing.

I have tried multiple approaches isolating the text as an object then using cat() to replace it after filtering, but the process must end with a CSV, which I have been unsuccessful achieving.


Viewing all articles
Browse latest Browse all 12111

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>