Logging error in Python 3.x : TypeError: a bytes-like object is required, not 'str'

Question

I am on Python 3.6.5.

While using logging I am getting the following error -

"TypeError: a bytes-like object is required, not 'str'"

It worked fine in Python 2.x and also I tried converting the string to Byte object but could not fix the problem

if __name__ == '__main__':
    config_file = '/source/account_content_recommendation/config/sales_central_config.json'
    try:
        ### Read all the parameters -
        params = json.loads(hdfs.read_file(config_file))

        ### Create the logging csv file -
        hdfs_log_path = params["hdfs_log_path"]
        hdfs.create_file(hdfs_log_path, "starting ... ", overwrite = True)
        log_name = 'Account_Content_Matching'


        global stream
        log = logging.getLogger('Acct_Cont_Log')
        stream = BytesIO()
        handler = logging.StreamHandler(stream)
        log.setLevel(logging.DEBUG)

        for handle in log.handlers:
            log.removeHandler(handle)

        log.addHandler(handler)

        #env = sys.argv[1]
        env = 'dev'
        formatter = logging.Formatter('{0}| %(asctime)s| {1}| %(module)s| %(funcName)s| %(lineno)d| %(levelname)s| %(message)r'.format(log_name, env))
        handler.setFormatter(formatter)

        log.info("starting execution of Account_Content_Matching load")
        #log.info("sys args %s"%(str(sys.argv)))

        def flush_log():
            global stream
            msg = stream.getvalue()
            hdfs.append_file(hdfs_log_path, msg)
            stream.seek(0)
            stream.truncate(0)
            print(msg)
            sys.stdout.flush

    except Exception as error:
        raise error

I am getting the below error -

Traceback (most recent call last): File "/home/ec2-user/anaconda3/envs/tensorflow_p36/lib/python3.6/logging/init.py", line 994, in emit stream.write(msg) TypeError: a bytes-like object is required, not 'str'

Also ...

Message: 'starting execution of Account_Content_Matching load' Arguments: () I1001 06:29:35.870266 140241833649984 :29] starting execution of Account_Content_Matching load

Answer 1

In your logger setup, you're:

• Using BytesIO as a stream
• Passing strings to it

This doesn't work, the 2 must be in sync. There are 2 ways to fix it. Either:

• Use [Python.Docs]: class io.StringIO(initial_value='', newline='\n') (instead of BytesIO):

  stream = StringIO()
  

• Convert all strings to bytes before passing them to the logger methods (more complicated, and makes less sense than the former):

  log.info("starting execution of Account_Content_Matching load".encode())  # log.info(b"starting execution of Account_Content_Matching load")  # For literals
  log.debug(some_string_var.encode())