Python pitfalls

Python 書籍

• Effective Python
• Python Cookbook
• Fluent Python
• Black Hat Python
• Gray Hat Python

關於參數傳遞

要小心 mutable 型態的資料類型, 像是 list, dictionary

def transmogrify(x):
    x[0] = 999
    # x = [4, 5, 6] 若指定新的物件則不會
    return x

x = [1, 2, 3]
print x
print transmogrify(x)
print x

[1, 2, 3]
[999, 2, 3]
[999, 2, 3]

小心預設參數傳遞的定義

Never give an empty list or other mutable structure as a default.

def f(x=[]):
    x.append(1)
    return x

print f()
print f()
print f()
print f(x=[9, 9, 9])
print f()
print f()

[1]
[1, 1]
[1, 1, 1]
[9, 9, 9, 1]
[1, 1, 1, 1]
[1, 1, 1, 1, 1]

通常我們會這麼修正, 當預設為空的時候進行檢查

def f(x = None):
    if x is None:
        x = []
    x.append(1)
    return x

print f()
print f()
print f()
print f(x = [9,9,9])
print f()
print f()

[1]
[1]
[1]
[9, 9, 9, 1]
[1]
[1]

map, filter, reduce

def square(x):
    return x ** 2
def cube(x):
    return x ** 3
print map(square, range(5))

def is_even(x):
    return x % 2 == 0
print filter(is_even, range(5))

def my_add(x, y):
    return x + y
print reduce(my_add, range(1, 6))

def custom_sum(xs, transform):
    return sum(map(transform, xs))
xs = range(5)
print xs
print custom_sum(xs, square)
print custom_sum(xs, cube)

[0, 1, 4, 9, 16]
[0, 2, 4]
15
[0, 1, 2, 3, 4]
30
100

closure

def make_logger(target):
    def logger(data):
        with open(target, 'a') as f:
            f.write(data + '\n')
    return logger
foo_logger = make_logger('foo.txt')
foo_logger('Hello')
foo_logger('World')

cat foo.txt
Hello
World

匿名函數

print map(lambda x: x ** 2, range(5))

[0, 1, 4, 9, 16]

其他

print reduce(lambda x, y: x + y, map(lambda x: x**2, range(1, 10)))

285

Recursion is used to show off the divide-and-conquer paradigm

def almost_quick_sort(xs):
    if xs == []:
        return xs
    else:
        pivot = xs[0]
        less_than = [x for x in xs[1:] if x <= pivot]
        more_than = [x for x in xs[1:] if x > pivot]
        return almost_quick_sort(less_than) + [pivot] + almost_quick_sort(more_than)
xs = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 9]
print almost_quick_sort(xs)

# [1, 1, 2, 3, 3, 4, 5, 5, 5, 6, 9, 9]

Iterators

由於一次只取一個值, 較不消耗記憶體, 對處理大數據較有助益

xs = [1, 2, 3]
x_iter = iter(xs)

print x_iter.next()
print x_iter.next()
print x_iter.next()
print x_iter.next()

1
2
3
Traceback (most recent call last):
  File "/Users/user/IPN/sandbox.py", line 7, in <module>
    print x_iter.next()
StopIteration

比較好的作法

x_iter = iter(xs)
for x in x_iter:
    print x

1
2
3

Generators

def count_down(n):
    for i in range(n, 0, -1):
        yield i
counter = count_down(10)
print counter.next()
print counter.next()
for count in counter:
    print count,

10
9
8 7 6 5 4 3 2 1

Iterators 可以使用下列像 list comprehension 的方式產生

xs2 = (x**2 for x in range(5))
print xs2

for x in xs2:
    print x,
print

<generator object <genexpr> at 0x1028abf50>
0 1 4 9 16

Iterators can be used for infinte functions

def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

for i in fib():
    if i > 1000: # 要記得設停止條件
        break
    print i,

0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

檔案處理傳回的方式就是 Iterator, 所以可以處理大資料

for line in open('foo.txt'):
    print line,

Generators and comprehensions

# A generator expression
print (x for x in range(10))

# A list comprehension
print [x for x in range(10)]

# A set comprehension
print {x for x in range(10)}

# A dictionary comprehension
print {x: x for x in range(10)}

<generator object <genexpr> at 0x1130d0960>
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
{0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9}

用 enumerate, 在做 loop 時, 不需要特別去記錄 index

xs = [1, 2, 3, 4]
for i, x in enumerate(xs):
    print i, x
    
0 1
1 2
2 3
3 4

當要 loop 多個相互配對的 list 時, zip 是很好的工具

當最短的 list 到底時就停止

xs = [1, 2, 3, 4]
ys = [10, 20, 30, 40]
zs = ['a', 'b', 'c', 'd', 'e']

for x, y, z in zip(xs, ys, zs):
    print x, y, z
    
1 10 a
2 20 b
3 30 c
4 40 d

做 list comprehension, 可用 ternary if-else operator

print [x**2 if x % 2 == 0 else x**3 for x in range(10)]
print [x**2 for x in range(10) if x % 2 == 0]

[0, 1, 4, 27, 16, 125, 36, 343, 64, 729]
[0, 4, 16, 36, 64]

Decorators

常見用法:

• logging
• profiling
• Just-In-Time (JIT) compilation

def func_timer(func):
    def f(*args, **kwargs):
        import time
        start = time.time()
        results = func(*args, **kwargs)
        print "Elapsed: %.2fs" % (time.time() - start)
        return results
    return f


@func_timer
def sleepy(msg, sleep=1.0):
    import time
    time.sleep(sleep)
    print msg

sleepy("Hello", 1.5)

Hello
Elapsed: 1.50s

operator

import operator as op

print reduce(op.add, range(10))
print reduce(op.mul, range(1, 10))

45
362880

排序時就很有用

import operator as op

my_list = [('a', 1), ('bb', 4), ('ccc', 2), ('dddd', 3)]
print sorted(my_list)
print sorted(my_list, key=op.itemgetter(1))
print sorted(my_list, key=lambda x: len(x[0]), reverse=True)

[('a', 1), ('bb', 4), ('ccc', 2), ('dddd', 3)]
[('a', 1), ('ccc', 2), ('dddd', 3), ('bb', 4)]
[('dddd', 3), ('ccc', 2), ('bb', 4), ('a', 1)]

python -c "from pkg_resources import Environment;print(Environment()['pip'])"