A year of waiting - and we received Python 3.12. Changes, innovations and additions

Many developers have been eagerly awaiting the new version of Python. Some people didn’t even wait, since the capabilities of the previous version were quite sufficient. But Python 3.12 was released, and now it has become known that this branch will be supported for a year and a half, and then patches will be generated for it to eliminate vulnerabilities – for another three and a half years.

In addition, the process of alpha testing of Python 3.13 has been launched, where a CPython build mode without a global interpreter lock (GIL, Global Interpreter Lock) is declared. The branch will be tested for seven months, correcting errors and adding new features. Then another three months – testing beta versions and after another two months the pre-final version will appear. But that will come later, and now let’s talk about what we already have in our hands – about Python 3.12.

So what did we get?

It is clear that during the year no one sat idly by – a lot of new and interesting things appeared in the new version. Here is a list of the most noticeable changes:

The resource efficiency of multi-core systems has been improved by adding support for isolated sub-interpreters and separate global locks (GIL, Global Interpreter Lock) for different interpreters within a process (CPython allows you to run several interpreters at once in one process). True, this new product is now available via the C-API, and support for the Python API will be added in the next branch.

Also added support for the Linux perf kernel subsystem in the interpreter. It makes it possible to determine the names of Python functions during profiling using the perf utility (previously, only the names of C functions were determined in traces).

Another important change is making error messages more informative. In addition, the range of exceptions with recommendations for eliminating typos has now been expanded. Here’s an example:

   sys.version_info
   NameError: name 'sys' is not defined. Did you forget to import 'sys'?

   somethin = blech
   NameError: name 'blech' is not defined. Did you mean: 'self.blech'?

   import a.y.z from b.y.z
   SyntaxError: Did you mean to use 'from ... import ...' instead?

   from collections import chainmap
   ImportError: cannot import name 'chainmap' from 'collections'. Did you mean: 'ChainMap'?

Active work on performance optimization also continues. It is moving little by little and there are first results – according to the announcement, productivity has become 5% higher. Not God knows what, but not bad either. The increase was achieved thanks to:

— Adding support for the BOLT binary optimizer, which improves performance by 1-5%.

— Added inline expansion of list inclusions (comprehensions). This innovation made it possible to speed up work with list inclusions.

— The size of Unicode objects has been reduced by 8-16 bytes.

— Increased speed of operations with regular expressions re.sub(), re.subn() and re.Pattern.

— The execution of isinstance() checks for some protocols has been accelerated – from 2 to 20 times.

– The tokenize.tokenize() and tokenize.generate_tokens() functions have also been accelerated.

A more compact type annotation syntax for generic classes and functions has been proposed. Here’s an example:

  def max[T](args: Iterable[T]) -> T:
       ...

   class list[T]:
       def __getitem__(self, index: int, /) -> T:
           ...

       def append(self, element: T) -> None:

There is a new way to define type aliases using the type expression. Another example:

   type Point = tuple[float, float]
   type Point[T] = tuple[T, T]

A new override decorator has also been added to the typing module. It informs type checking systems that a method in a subclass is intended to override a method or attribute in the superclass. And one more example:

   class Base:
       def log_status(self) -> None:
           ...
   
   class Sub(Base):
       @override
       def log_status(self) -> None:  # Ok, переопределяет Base.log_status
           ...

       @override
       def done(self) -> None:  # Система проверки типов выявит ошибку
           …

The new version increases the flexibility of parsing f-strings (formatted literals prefixed with ‘f’), which opens up new possibilities, including the ability to specify any expressions valid in Python, including multiline expressions, comments, backslashes, and Unicode escape sequences. In an internal string, you can reuse double quotes without using single quotes. The information content of messages in f-lines has also been increased, in which the exact place in the line that caused the error is now indicated. Here’s an example:

   print(f"This is the playlist: {"\n".join(songs)}")

   print(f"This is the playlist: {"\N{BLACK HEART SUIT}".join(songs)}")

   print(f"This is the playlist: {", ".join([
   ...     'Take me back to Eden',  # My, my, those eyes like fire
   ...     'Alkaline',              # Not acid nor alkaline
   ...     'Ascensionism'           # Take to the broken skies at last
   ... ])}")

Security has also been improved. Thus, the built-in implementations of the SHA1, SHA3, SHA2-384, SHA2-512 and MD5 algorithms in hashlib have been replaced with formally verified options from the HACL* project (the built-in implementations are used only if OpenSSL is not available).

Added stack overflow protection to Cpython.

The pathlib.Path module now supports subclassing, and the os module now has expanded Windows support. In particular, support for the os.listdrives(), os.listvolumes() and os.listmounts() methods has been added for this OS, and the accuracy of os.stat() and os.lstat() has also been improved.

In addition, command line interfaces (“python -m sqlite3” and “python -m uuid”) have been added to the sqlite3 and uuid modules.

In addition to new features, the developers decided to clean up some old ones that are no longer needed. In particular, the asynchat, asyncore, smtpd, imp and distutils modules have been removed (the distutils module can be used from the setuptools package). Removed obsolete methods in unittest. We also got rid of outdated and incorrectly working functions, classes and methods. First of all, these are locale.format(), io.OpenWrapper, ssl.RAND_pseudo_bytes(), ElementTree.Element.copy(), hashlib.pbkdf2_hmac(), gzip.GzipFile.