I’ve been working with Python on and off for more than ten years now. Along the way I’ve gathered a few small tips and tricks for Python programming. Most of them didn’t feel like they warranted their own blog post, so I’ve compiled them into a single entry instead.
If you’ve got some of your own that you’d like to share I’m on Twitter as @therealchreke—don’t be a stranger!
Functions
Named arguments
Positional arguments can be used as named arguments, i.e. you can use the same a="b" syntax as you would for a keyword argument. Here’s an example:
def foo(a, b):
return [a, b]
foo(b="B", a="A")
# ["A", "B"]
Useful for calling functions where it’s not obvious which positional argument is which.
Identity
Some of the built-in constructor functions, e.g. str, int, float etc, will return an identity value when called without arguments.
For example, str() will return the empty string, int() will return 0 and so on. This isn’t terribly useful by itself, but can come in handy when you need a function that will return a “default” value; we’ll see some examples of this further down.
Omitting parentheses
You can omit parentheses from certain expressions (like tuples). For example, parentheses can be omitted from generator comprehensions if the comprehension is already parenthesised:
# These two are equivalent:
sum((x ** 2 for x in range(3)))
sum(x ** 2 for x in range(3))
Generators
Yield from
Since Python 3.3 it’s possible to use yield from to yield directly from another generator.
# These two are equivalent:
def foo():
for i in some_iterable:
yield i
def foo():
yield from some_iterable
Creating generators with iter()
iter(iterable) creates a generator from an iterable.
iter(callable, sentinel) creates a generator that returns calls from the (zero-argument) callable until it returns the sentinel value. This means you can easily create an infinite generator by doing something like iter(int, None) This generator will yield an infinite number of zeroes, as int() returns 0 (see above) which will never be equal to None
Dictionaries
defaultdict
Have you ever wanted your Python dictionaries to automatically create keys when you try to access them? For example:
d = {}
# This throws a KeyError since the key "foo" is not set:
d["foo"]["bar"] = "baz"
Now you can!
from collections import defaultdict
def makehash():
return defaultdict(makehash)
d = makehash()
d["foo"]["bar"] = "baz"
(For the record I think this is a terrible idea, but there you go)
Checking if a dict a subset of another dict
Since dict.items acts as a set you can use the overloaded “ operator to check if a dict is a “subset” of another dict:
{"a": 1, "b": 2}.items() > {"b": 2}.items()
# True
Dates and times
If you’re working with human (i.e. weird) dates and times, use the [calendar][calendar] module to maintain your sanity.
Durations
Instead of using magic numbers for durations of time, e.g.
# 12 hours
ttl_in_seconds = 60 * 60 * 12
use the built-in timedelta functions and types:
ttl = datetime.timedelta(hours=12)
expires_in = datetime.datetime.now() + ttl
This gives you safety at both the type and unit level, e.g. there’s no risk of confusing seconds and milliseconds.
Enums
If some input should be constrained to a set of values, use the enum module, e.g.
from enum import Enum
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
Enum values can be accessed either by dot notation or subscripts, e.g:
Color.RED == Color['RED']
To convert a primitive value to an enum instance, use the Enum‘s constructor:
Color("RED")
Note that an enum instance is not equal to its corresponding primitive value!
Color.RED != "RED"
Remember that enums are regular classes; this can be used e.g. to provide utility methods for working with enum values.
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
def hex_color(self):
hex_values = {
"RED": "#ff0000",
"GREEN": "#00ff00",
"BLUE": "#0000ff",
}
return hex_values[self.name]
Numbers
If you need a really big number, why settle for anything less than infinity?
float('Inf')
float('-Inf')
(This is actually part of the IEEE 754 spec)
Decimal
If you need precise arithmetic when dealing with fractions, use the decimal module. Your accountant will thank you.
A neat trick when working with numbers in JSON is that they’re represented as arbitrary-precision decimal strings. This means they can be losslessly converted to and from Decimal instances, as in this example from the JSON module documentation:
json.loads('1.1', parse_float=decimal.Decimal)
Tip: simplejson comes with out-of-the-box support for using decimal for parsing and serialization.
Metaprogramming
You can use the inspect module to get information about live objects, e.g. the argument names of a function. Useful, for example, when you want to decorate a function and tap into one of its arguments.
You can also use inspect.getdoc to get the docstring of a function at runtime.
Shell Utilites
Python has some cool convenience utilities:
Start a HTTP server that serves files from the current directory:
python3 -m http.server
Pretty-print JSON:
python3 -m json.tool < foo.json
You can use pydoc or pydoc3 to read Python documentation on the command line.
Modules and packages
Referencing imports
Whenever you try to use a Python mocking tool you may be surprised by the following behaviour: If you use from X import Y then Y will be added to the importing module’s namespace.
Let’s say a module foo imports bar.quux the following way:
from bar import quux
Then the way to reference the quux module at runtime becomes foo.quux and not bar.quux as one might expect.
Packages
The presence of an __init__.py file (which may be empty) indicates that a directory is a Python package.
The file may be empty, but can also contain executable code. One common trick to consolidate multiple files into what appears to be one module is to do this in __init__.py
# In file: foo/__init__.py
from foo import Foo
from bar import Bar
# Now you can import Foo and Bar like so:
from foo import Foo, Bar
Gotcha: Remember that every parent directory of the package path must also contain an __init__.py file! Forgetting to include this file in some directory can break imports and static analysis tools.
Code style
I used to have a lot of opinions about how to best format Python code, but now I have only one: Use black.
Misc
Python is one of the few languages that allow chained use of the lesser- / greater-than operators:
if 0 < foo < 10:
do_something()
Gotchas
Default keyword arguments
Default keyword argument values are not re-instantiated every time a method is called call, e.g. if you have a kwarg foo={} then every call to that function will refer to the same dictionary. Oops!
def foo(a=[]):
return a.append("test")
foo() # Returns ["test"]
foo() # Returns ["test", "test"]
To avoid surprising behaviour, don’t use a mutable types as default arguments; use None instead.