02 Mar 2024

Unlocking Python’s Hidden Powers: A Deep Dive into Special Methods

For someone who has come over to Python programming from other languages, Python special methods also referred to as dunder methods (example __init__(self)), are fascinating but don’t come readily to the forefront of our code designs when working on a Python codebase.

This is very true if you have spent a long time building software in traditional OOP languages like Java, C# and PHP, everything is typically designed around the four pillars of OOP principles - Abstraction, Encapsulation, Inheritance and Polymorphism. TLDR, design using a class and inherit from a superclass et al.

I was recently reading an article on Data Structure and the sample code was implemented in Ruby, I went ahead to translate the implementation from Ruby to Python. There came a point when I needed to return a list of Nodes and in typical OOP fashion, I wrote a get_nodes method to align with the Ruby implementation. I remembered Python has numerous special methods of which the __iter__() special method for iterating and returning items on demand.

For my learning and note-taking experience, I have decided to take a closer look at Python’s special methods (from here on out, I will refer to them as dunder methods), and surface useful ones I could be using consciously in my day-to-day as well as explore ones I don’t use often but could come in handy at some point.

What are Python’s special methods?

From Python’s documentation, a special method is defined as

A method that is called implicitly by Python to execute a certain operation on a type, such as addition. Such methods have names starting and ending with double underscores.

Reference: https://docs.python.org/3/glossary.html#term-special-method

A common Python special method for anyone writing a tangible piece of Python code is __init__(self) and this translates into constructors in other languages. However, python has a plethora of special methods that have been grouped into the following categories just to name a few:

I will be cherry-picking a few dunder methods I find interesting and highlighting scenarios that demonstrate how a typical code would be written and later proceed to demonstrate how a dunder method implementation would look.

The `iter()` method

The __iter__() method makes an appearance when working with iterables in python. When working with data structures that can hold a collection of values like lists, sets, sequence etc.

Now in Python land, there are Iterables and Iterators. Iterables typically implement the __iter__ method and return items on demand. However, Iterators, implement __iter__ that typically returns self but also implement a __next__() method for fetching the next value in the squence.

Also, you can implement iterables using a class-based approach by extending the typing.Iterable class and implementing the abstract __iter__ method. There is also an accompanying typing.iterator class if what you seek is an iterator.

If you want to turn a class into an iterable class that can be looped over for example, then you have to write an __iter__() method and yield items one at a time typically from within a loop.

As an example, let us write a simple StudentCollections class that does one simple task of holding a list of students. To keep things simple, I will store the student names and not do anything fancy. This is just an example after all but you get the idea.

Typical approach

class StudentCollection:

  def __init__(self, students: list[str] | None = None):
    self._students = students

  def add_student(self, student: str):
    if self._students is None:
      self._students = []
    self._students.append(student)

  def list_all_students(self) -> list[str] | None:
    return self._students

The above code works and is readable. I have written many a code like the above. The code below shows how one would use the above class to hold a list of student names and print them out.

# could optionally have initialised with an array of students
student_names = ['student1', 'student2', 'student3']

collection = StudentCollection(student_names)
print(collection.list_all_students())

# output : ['student1', 'student2', 'student3']

Using `iter()` method

class StudentCollection:

  def __init__(self, students: list[str] | None = None):
    self._students = students

  def add_student(self, student: str):
    if self._students is None:
      self._students = []
    self._students.append(student)

  # we can remove the list_all_students() method.

  def __iter__(self) -> Generator[str | None, None, None]:
    if self._students:
      for student in self._students:
        yield student

The change made to the StudentCollection class was to remove the list_all_students method and implement the __iter__ method.

Using the new code looks cleaner and lends itself to utilizing built-in Python functions that work with iterables like list(), set()

student_names = ['student1', 'student2', 'student3']
collection = StudentCollection(student_names)

# you could fetch all the students if you just needed a list
print(list(collection))

# You can also loop through each item and do some work
for student in collection:
    do_something(student)

Final example on __iter, let’s say we happen to have duplicate student names in our StudentCollection instance, but our program needs a unique list/set, implementing the __iter__ allows us to write cleaner more concise Python code.

# 'student1' has been added twice to the list. We don't want that.
student_names = ['student1', 'student2', 'student3', 'student1']
collection = StudentCollection(student_names)

# You can make this a list rather than a set by writing `list(set(collection))`
unique_names = set(collection)

print(unique_names)
# Output: {'student1', 'student2', 'student3'}.

Summary

Adopting the __iter__ method will help you utilize built-in capabilities within the Python language. It also lends itself to writing cleaner and dare I say more pythonic code. A lot of Python libraries make heavy use of iterables and having a look will give you more ideas on where to employ them.