Line continuation for list comprehensions or generator expressions in python

Question

How are you supposed to break up a very long list comprehension?

[something_that_is_pretty_long for something_that_is_pretty_long in somethings_that_are_pretty_long]

I have also seen somewhere that people that dislike using '\' to break up lines, but never understood why. What is the reason behind this?

Answer 1

[x
 for
 x
 in
 (1,2,3)
]

works fine, so you can pretty much do as you please. I'd personally prefer

 [something_that_is_pretty_long
  for something_that_is_pretty_long
  in somethings_that_are_pretty_long]

The reason why \ isn't appreciated very much is that it appears at the end of a line, where it either doesn't stand out or needs extra padding, which has to be fixed when line lengths change:

x = very_long_term                     \
  + even_longer_term_than_the_previous \
  + a_third_term

In such cases, use parens:

x = (very_long_term
     + even_longer_term_than_the_previous
     + a_third_term)

Answer 2

You can also make use of multiple indentations in cases where you're dealing with a list of several data structures.

new_list = [
    {
        'attribute 1': a_very_long_item.attribute1,
        'attribute 2': a_very_long_item.attribute2,
        'list_attribute': [
            {
                'dict_key_1': attribute_item.attribute2,
                'dict_key_2': attribute_item.attribute2
            }
            for attribute_item
            in a_very_long_item.list_of_items
         ]
    }
    for a_very_long_item
    in a_very_long_list
    if a_very_long_item not in [some_other_long_item
        for some_other_long_item 
        in some_other_long_list
    ]
]

Notice how it also filters onto another list using an if statement. Dropping the if statement to its own line is useful as well.

Answer 3

I'm not opposed to:

variable = [something_that_is_pretty_long
            for something_that_is_pretty_long
            in somethings_that_are_pretty_long]

You don't need \ in this case. In general, I think people avoid \ because it's slightly ugly, but also can give problems if it's not the very last thing on the line (make sure no whitespace follows it). I think it's much better to use it than not, though, in order to keep your line lengths down.

Since \ isn't necessary in the above case, or for parenthesized expressions, I actually find it fairly rare that I even need to use it.

Answer 4

I would argue that you shouldn't use a list comprehension in this case, and instead use a for loop:

result = []
for something_that_is_pretty_long in somethings_that_are_pretty_long:
    result.append(something_that_is_pretty_long)

One reason to use list comprehensions over a for loop + .append() is that it can be much more concise than using an explicit for loop. However, when the list comprehension needs to be split over multiple lines, that conciseness can make the expression extremely difficult to read.

While PEP8 doesn't explicitly forbid multi-line list comprehensions, the Google Python Style Guide requires that each portion of a list comprehension fit on a single line (emphasis mine):

2.7 Comprehensions & Generator Expressions

Okay to use for simple cases. Each portion must fit on one line: mapping expression, for clause, filter expression. Multiple for clauses or filter expressions are not permitted. Use loops instead when things get more complicated.