This is a collection of more or less exotic Python tips and tricks I’ve gathered over the years.
If you’ve got some of your own that you’d like to share hit me up on Twitter!
Functions
Positional arguments can be used as named arguments, i.e. you can use the same
a="b"
syntax as you would for a keyword argument. Here’s an example:
def foo(a, b):
return [a, b]
foo(b="B", a="A")
# ["A", "B"]
Useful for calling functions where it’s not obvious which positional argument is which.
Comprehensions
There are a variety of comprehensions in Python:
- List:
[x for x in iterable]
- Dict:
[k: v for k, v in iterable]
- Set:
{x for x in iterable}
- Generator:
(x for x in iterable)
Comprehensions can contain multiple for
clauses. The following yields the
“cross product” of list1
and list2
:
numbers = [1, 2]
letters = ["a", "b"]
[(n, l) for n in numbers for l in letters]
# [(1, "a"), (2, "a"),
# (1, "b"), (2, "b")]
The output of one for
clause can also be used as input to another for
clause. To flatten a nested list, for example, you can do this:
nested = [(1, 2), (3, 4), (5, 6)]
[b for a in nested for b in a]
# [1, 2, 3, 4, 5, 6]
(Mnemonic: The comprehension goes from left-to-right, outside-in)
Generator expressions
Generator expressions are like list comprehensions, but return a generator instead of a list.
Cool trick: Just like with tuples, parentheses can be omitted from generator comprehensions if the expression is already parenthesised:
# These two are equivalent:
sum((x ** 2 for x in range(3)))
sum(x ** 2 for x in range(3))
Generators
Yield from
Since Python 3.3 it’s possible to use yield from
to yield directly
from another generator.
# These two are equivalent:
def foo():
for i in some_iterable:
yield i
def foo():
yield from some_iterable
Creating generators with iter
iter(iterable)
creates a generator from an iterable.
iter(callable, sentinel)
creates a generator that returns calls from the
(zero-argument) callable until it returns the sentinel
value. This means you
can easily create an infinite generator by doing something like iter(int,
None)
(since int()
returns 0
).
Dictionaries
Have you ever wanted your Python dictionaries to automatically create keys when you try to access them? For example:
d = {}
# This throws a KeyError since the key "foo" is not set:
d["foo"]["bar"] = "baz"
Now you can!
from collections import defaultdict
def makehash():
return defaultdict(makehash)
d = makehash()
d["foo"]["bar"] = "baz"
(For the record I think this is a terrible idea, but there you go)
Checking if a dict a subset of another dict
Since dict.items
acts as a set
you can use the overloaded >
operator to
check if a dict is a “subset” of another dict:
{"a": 1, "b": 2}.items() > {"b": 2}.items()
# True
Dates and times
If you’re working with human (i.e. weird) dates and times, use the calendar module to maintain your sanity.
Durations
Instead of using magic numbers for durations of time, e.g.
# 12 hours
ttl_in_seconds = 60 * 60 * 12
use the built-in timedelta
functions and types:
ttl = datetime.timedelta(hours=12)
expires_in = datetime.datetime.now() + ttl
This gives you safety at both the type and unit level, e.g. there’s no risk of confusing seconds and milliseconds.
Enums
If some input should be constrained to a set of values, use the enum
module, e.g.
from enum import Enum
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
Enum values can be accessed either by dot notation or subscripts, e.g:
Color.RED == Color['RED']
To convert a primitive value to an enum instance, use the Enum
‘s constructor:
Color("RED")
Note that an enum instance is not equal to its corresponding primitive value!
Color.RED != "RED"
Remember that enums are regular classes (well, for the most part); this can be used e.g. to provide utility methods for working with enum values.
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
def hex_color(self):
hex_values = {
"RED": "#ff0000",
"GREEN": "#00ff00",
"BLUE": "#0000ff",
}
return hex_values[self.name]
Numbers
If you need a really big number, why settle for anything less than infinity?
float('Inf')
float('-Inf')
(This is actually part of the IEEE 754 spec)
Decimal
If you need precise arithmetic when dealing with fractions, use the decimal module. Your accountant will thank you.
A neat trick when working with numbers in JSON is that they’re represented as
arbitrary-precision decimal strings. This means they can be losslessly
converted to and from Decimal
instances, as in this example from the JSON
module documentation:
json.loads('1.1', parse_float=decimal.Decimal)
Tip: simplejson comes with out-of-the-box support for using
decimal
for parsing and serialization.
Metaprogramming
You can use the inspect module to get information about live objects, e.g. the argument names of a function. Useful, for example, when you want to decorate a function and tap into one of its arguments.
You can also use inspect.getdoc
to get the docstring of a function at
runtime.
Shell Utilites
Python has some cool convenience utilities:
Start a HTTP server that serves files from the current directory:
python3 -m http.server
Pretty-print JSON:
python3 -m json.tool < foo.json
You can use pydoc
or pydoc3
to read Python documentation on the command
line.
Modules and packages
Referencing imports
Whenever you try to use a Python mocking tool you may be surprised by the
following behaviour: If you use from X import Y
then Y
will be added to the
importing module’s namespace.
Let’s say a module foo
imports bar.quux
the following way:
from bar import quux
Then the way to reference the quux
module at runtime becomes foo.quux
and
not bar.quux
as one might expect.
Packages
The presence of an __init__.py
file (which may be empty) indicates that a
directory is a Python package.
The file may be empty, but can also contain executable code. One common
trick to consolidate multiple files into what appears to be one module is to
do this in __init__.py
:
# In file: foo/__init__.py
from foo import Foo
from bar import Bar
# Now you can import Foo and Bar like so:
from foo import Foo, Bar
Gotcha: Remember that every parent directory of the package path must also
contain an __init__.py
file! Forgetting to include this file in some
directory can break imports and static analysis tools.
Code style
I used to have a lot of opinions about how to best format Python code, but now I have only one: Use black.
Misc
Python is one of the few languages that allow chained use of the lesser- / greater-than operators:
if 0 < foo < 10:
do_something()
Gotchas
Default keyword arguments
Default keyword argument values are not re-instantiated every time
a method is called call, e.g. if you have a kwarg foo={}
then every
call to that function will refer to the same dictionary. Oops!
def foo(a=[]):
return a.append("test")
foo() # Returns ["test"]
foo() # Returns ["test", "test"]
To avoid surprising behaviour, don’t use any of Python’s mutable
types as default arguments; use None
instead.