Writing a Luigi Target as a csv with Pandas

Question

I have a basic Luigi pipeline that I'm writing. The pipeline will download Apple stock data and create a CSV out of it. The following is what I've written:

# Download Apple data
class DownloadSymbol(luigi.Task):
    symbol = luigi.Parameter()

    def output(self):
        return luigi.LocalTarget(f'data/{symbol.lower()}_data.csv')
    
    def run(self):
        df = download_symbol(self.symbol)
        with self.output().open('w') as csv:
            df.to_csv(csv)

I wrote the context manager based on this article. But when I run this pipeline, I receive TypeError: write() argument must be str, not bytes. I've tried changing the context manager to be a wb-type write, but I receive the same error.

How do I correctly utilize pd.DataFrame.to_csv() in a Luigi pipeline?

Answer 1

This should work

with self.output().open('w') as csv: 
    csv.write(df.to_csv())