Fixing SimpleLazyObject Errors in Django 3.1

By Aaron O. Ellis

Friday, August 21, 2020

The Quick Fix

int() argument must be a string, a bytes-like object or a number, not 'SimpleLazyObject'.

If you’re seeing the above TypeError in Django 3.1, you are likely doing something like this in a View class with context data:

class ExampleView(TemplateView):
    def get_context_data(self, **kwargs):
        get_object_or_404(MyObject, pk=kwargs.get('pk'))

This error occurs even if the pk parameter is converted to an int in the path, e.g. path('<int:pk>', ....

To fix, use self.kwargs instead:

class ExampleView(TemplateView):
    def get_context_data(self, **kwargs):
        get_object_or_404(MyObject, pk=self.kwargs.get('pk'))

You’ll see the same TypeError when just calling int() directly on the kwargs parameter, such as int(kwargs.get('pk')).

You may also see django.db.utils.ProgrammingError if you’re passing any of the keyword arguments to a database model. There is an ongoing ticket for this issue.

Using self.kwargs (or view.kwargs in templates) is the change you’d have to make by Django 4.0 anyway, since URL parameter keyword arguments will no longer be passed directly to the context. The underlying issue is caused by a mishandled deprecation warning, which unfortunately has no easy fix.

A More Detailed Explanation

Class-based views were introduced in Django 1.3 and included the highly useful TemplateView. Originally, the TemplateView passed any matched URL parameters to its template context data under the key params, allowing the user to access these parameters in the templates without any additional coding.

With the introduction of the ContextMixin in Django 1.5, the URL parameters were added directly to the template context. For example, the named URL parameter item could be accessed in templates with {{ item }}. This was done by having the TemplateView pass kwargs directly to the inherited get_context_data method:

def get(self, request, *args, **kwargs):
    context = self.get_context_data(**kwargs)

But the introduction of ContextMixin created an issue that persists today: the View instance wants to add itself to the context, but will be prevented from doing so if there is a URL parameter named view.

def get_context_data(self, **kwargs):
    if 'view' not in kwargs:
        kwargs['view'] = self

URL parameters have also been accessible in templates under the view’s kwargs property (if the view was correctly added to the context). For example with the URL parameter item: {{ view.kwargs.item }}.

Not until Django 3.1 was a change made to solve this issue. The direct passing of URL kwargs to the context is now deprecated and will be removed in Django 4.0 (approximately 16 months from now). Users should just use the kwargs attached to the view instance.

To warn users of this deprecated feature in Django 3.1, the passed kwargs are wrapped in a closure and transformed into a SimpleLazyObject (code edited for brevity):

context_kwargs = {}
for key, value in url_kwargs.items():
    def access_value(key=key, value=value):
        warnings.warn('<warning message')
        return value
    context_kwargs[key] = access_value

SimpleLazyObject is a creation of the Django project and lives in its functional utilities. When a variable matching the passed URL parameter is called in a template, the SimpleLazyObject behaves as intended, rendering the parameter and issuing a deprecation warning (which you can enable by running the process with python -Wa):

RemovedInDjango40Warning: TemplateView passing URL kwargs to the context is deprecated. Reference item in your template through view.kwargs instead.
self._wrapped = self._setupfunc()

The SimpleLazyObject works in Django templates because the default behavior of template rendering is to call str() on any non-str variable types (plus some escaping), and SimpleLazyObject proxies this function to the inner value of its wrapped function. But SimpleLazyObject doesn’t know what to do with the built-in function int(), since this function is not proxied. The lack of this proxy method also explains the failure of get_object_or_404() when querying with an integer primary key, since int() is used during prep value.

And while adding __int__ = new_method_proxy(int) to SimpleLazyObject would solve some of our issues, model managers still wouldn’t know what to do with these objects, causing errors such as: Error binding parameter - probably unsupported type or django.db.utils.ProgrammingError: can't adapt type '__proxy__'.

Wrapping URL parameter values was likely a bad idea, even if the goal was to indicate an upcoming deprecation. URL parameters are mostly built-in types, and converting them to lazy-evaluated objects is asking for trouble. It makes more sense to wrap accessor methods on the context dictionary, but the structure of Django’s ContextMixin makes it difficult if not impossible to add a deprecation to the parameter kwargs.

A bug like this gets through rigorous testing and QA because it is a perfect case of use-case blindness: the Django developers intended the passed URL parameters to be rendered in templates, but the code to enable that feature also allowed the parameters to be used anywhere in the context body. Any testing of the recent changes would pass, because only the intended use was tested, instead of all possible uses.

If testing all possible use cases sounds impossible, you’re right. Even a test suite with 100% coverage could miss this bug, since template rendering would successfully execute all involved lines of code.

If you’re building a framework for other users, expect them to invent new use cases for your code - cases you may even consider misuse. And if issues arise during this misuse, know that it was your code that enabled it. After all, there are really only two types of frameworks in the world: those that users will abuse, and those with no users.