Python etc
6.12K subscribers
18 photos
194 links
Regular tips about Python and programming in general

Owner — @pushtaev

© CC BY-SA 4.0 — mention if repost
Download Telegram
Creation of class instance is done by __call__ method of object class (provided by metaclass type) and practically includes only 2 steps:

1. Call the __new__ method to create an instance.
2. Call the __init__ method to set up the instance.

class A:
def __new__(cls, *args):
print('new', args)
return super().__new__(cls)

def __init__(self, *args):
print('init', args)

A(1)
# new (1,)
# init (1,)

A.__call__(1)
# new (1,)
# init (1,)


So, if you want to create an instance without executing __init__, just call __new__:

A.__new__(A, 1)                                                                         
# new (1,)


Of course, that's a bad practice. The good solution is to avoid a heavy logic in __init__ so nobody wants to avoid calling it.
Some functions can accept as an argument value of any type or no value at all. If you set the default value to None you can't say if None was explicitly passed or not. For example, the default value for argparse.ArgumentParser.add_argument. For this purpose, you can create a new object and then use is check:

DEFAULT = object()

def f(arg=DEFAULT):
if arg is DEFAULT:
return 'no value passed'
return f'passed {arg}'

f() # 'no value passed'
f(None) # 'passed None'
f(1) # 'passed 1'
f(object()) # 'passed <object object at ...>'


The module unittest.mock provides a sentinel registry to create unique (by name) objects for the testing purpose:

sentinel.ab.name # 'ab'
sentinel.ab is sentinel.ab # True
sentinel.ab is sentinel.cd # False
Accidentally, yield can be used in generator expressions and comprehensions:

[(yield i) for i in 'ab']
# <generator object <listcomp> at 0x7f2ba1431f48>

list([(yield i) for i in 'ab'])
# ['a', 'b']

list((yield i) for i in 'ab')
# ['a', None, 'b', None]


This is because yield can be used in any function (turning it into a generator) and comprehensions are compiled into functions:

>>> dis.dis("[(yield i) for i in range(3)]")                                                                                                                                             
0 LOAD_CONST 0 (<code object <listcomp> ...>)
2 LOAD_CONST 1 ('<listcomp>')
4 MAKE_FUNCTION 0
...


This produces a warning in Python 3.7 and will raise SyntaxError in python 3.8+. However, yield inside lambda still can be used:

a = lambda x: (yield x)
list(a(1))
# [1]
Docstring is a string that goes before all other statements in the function body (comments are ignored):

def f(): 'a'
f.__doc__ # 'a'

def f(): r'a'
f.__doc__ # 'a'


It must be a static unicode string. F-strings, byte-strings, variables, or methods can't be used:

def f(): b'a'
f.__doc__ # None

def f(): f'a'
f.__doc__ # None

a = 'a'
def f(): a
f.__doc__ # None

def f(): '{}'.format('a')
f.__doc__ # None


Of course, you can just set __doc__ attribute:

def f(): pass
f.__doc__ = f'{"A!"}'
f.__doc__ # 'A!'
Hi there!

My team and I are looking for new developers once again. We are making a voice assistant, which is currently growing rapidly. Also our first smart speaker is already released:
We can offer big data, highload, modern development techniques, bleeding-edge data science technologies and a really good team (and I mean it).

We expect you to know Python and be smart and cunning.

The team is based in Moscow but remote work is fully supported. Russian is required though.

If you can recommend your friend, we are ready to pay 100 000 roubles if he is hired.

Contact @pushtaev for any details.
The module codecs provides encode and decode function to encode and decode (wow!) text in different encodings, like UTF8, CP1251, Punycode, IDNA, ROT13, execute escape sequences, etc.

codecs.encode('hello, @pythonetc', 'rot13')
# 'uryyb, @clgubargp'

codecs.encode('\n', 'unicode_escape')
# b'\\n'

codecs.encode('привет, @pythonetc', 'punycode')
# b', @pythonetc-nbk5b4b7gra3b'

codecs.encode('привет, @pythonetc', 'idna')
# b'xn--, @pythonetc-nbk5b4b7gra3b'

codecs.encode('привет, @pythonetc', 'cp1251')
# b'\xef\xf0\xe8\xe2\xe5\xf2, @pythonetc'
As we said, comprehensions compiled into functions. That means, we can take a types.CodeType object for a comprehension, wrap it into types.FunctionType and get a function.

import types

def make():
[x*2 for x in _]

code = make.__code__.co_consts[1]
func = types.FunctionType(code, globals())

# call the function!
func(iter(range(5)))
# [0, 2, 4, 6, 8]
__slots__ can be used to save memory. You can use any iterable as __slots__ value, including dict. AND Starting from Python 3.8, you can use dict to specify docstrings for slotted attributes __slots__:

class Channel:
"Telegram channel"
__slots__ = {
'slug': 'short name, without @',
'name': 'user-friendly name',
}
def __init__(self, slug, name):
self.slug = slug
self.name = name

inspect.getdoc(Channel.name)
# 'user-friendly name'


Also, help(Channel) lists docs for all slotted attributes:

class Channel(builtins.object)
| Channel(slug, name)
|
| Telegram channel
|
| Methods defined here:
|
| __init__(self, slug, name)
| Initialize self. See help(type(self)) for accurate signature.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| name
| user-friendly name
|
| slug
| short name, without @
​​The script pydoc can be used to see documentationand docstrings from the console:

$ pydoc3 functools.reduce | cat
Help on built-in function reduce in functools:

functools.reduce = reduce(...)
reduce(function, sequence[, initial]) -> value

Apply a function of two arguments cumulatively to the items of a sequence,
...


Also, you can specify a port with -p flag, and pydoc will serve the HTML documentation browser on the given port:

$ pydoc3 -p 1234
Server ready at http://localhost:1234/
Server commands: [b]rowser, [q]uit
server>
PEP-560 (landed in Python 3.7) introduced a new magic method __class_getitem__. it is the same as __getitem__ but for not instancinated class. The main motivation is easier type annotation support for generic collections like List[int] or Type[Dict[str, int]]:

class MyList:
def __getitem__(self, index):
return index + 1

def __class_getitem__(cls, item):
return f"{cls.__name__}[{item.__name__}]"

MyList()[1]
# 2

MyList[int]
# 'MyList[int]'
ipaddress provides capabilities to work with IP addresses and networks (both IPv4 and IPv6).

ip = ipaddress.ip_address('127.0.0.1')
ip.is_private # True
ip.is_loopback # True
ip.is_global # False
ip.is_multicast # False
ip.is_reserved # False
ip.is_unspecified # False
ip.reverse_pointer # '1.0.0.127.in-addr.arpa'

net = ipaddress.ip_network('192.168.0.0/28')
net.is_private # True
net.hostmask # IPv4Address('0.0.0.15')
net.num_addresses # 16
In Python 2.5, PEP-357 allowed any object to be passed as index or slice into __getitem__:

class L:
def __getitem__(self, value):
return value

class C:
pass

L()[C]
# <class __main__.C ...>


Also, it introduced a magic method __index__. it was passed instead of the object in slices and used in list and tuple to convert the given object to int:

class C:
def __index__(self):
return 1

# Python 2 and 3
L()[C()]
# <__main__.C ...>

L()[C():]
# Python 2:
# slice(1, 9223372036854775807, None)
# Python 3:
# slice(<__main__.C object ...>, None, None)

# python 2 and 3
[1,2,3][C()]
# 2


The main motivation to add __index__ was to support slices in numpy with custom number types:

two = numpy.int64(2)

type(two)
# numpy.int64

type(two.__index__())
# int


Now it is mostly useless. However, it is a good example of language changes to meet the needs of a particular third-party library.
In Python 3.3, PEP-3155 introduced a new __qualname__ attribute for classes and functions which contains a full dotted path to the definition of the given object.

class A:
class B:
def f(self):
def g():
pass
return g

A.B.f.__name__
# 'f'

A.B.f.__qualname__
# 'A.B.f'

g = A.B().f()
g.__name__
# 'g'

g.__qualname__
# 'A.B.f.<locals>.g'
​​We use Arabic digits to record numbers. However, there are many more numeral systems: Chinese (and Suzhou), Chakma, Persian, Hebrew, and so on. And Python supports them when detecting numbers:

int('٤٢')
# 42

'٤٢'.isdigit()
# True

import re
re.compile('\d+').match('٤٢')
# <re.Match object; span=(0, 2), match='٤٢'>


If you want to match only Arabic numerals, make an explicit check for it:

n = '٤٢'
n.isdigit() and n.isascii()
# False

re.compile('[0-9]+').match(n)
# None


Let's make the full list of supported numerals:

from collections import defaultdict
nums = defaultdict(str)
for i in range(0x110000):
try:
int(chr(i))
except:
pass
else:
nums[int(chr(i))] += chr(i)
dict(nums)
Python has rich support for Unicode, including referencing glyphs (including emojis, of course) by name.

Get glyph name:

'🤣'.encode('ascii', 'namereplace')
# b'\\N{ROLLING ON THE FLOOR LAUGHING}'


Convert name to a glyph:

'\N{ROLLING ON THE FLOOR LAUGHING}'
# '🤣'

# case doesn't matter:
'\N{Rolling on the Floor Laughing}'
# '🤣'


A good thing is that f-string also don't confused by named unicode glyphs:

fire = 'hello'
f'{fire} \N{fire}'
# 'hello 🔥'