Hi everyone! I’m Eklavya, a Google Summer of Code student working on Zulip. This summer, I mostly worked on 2 things for Zulip:
Making Zulip’s code Python 3 compatible.
Adding mypy type annotations to Zulip’s python code.
It was a great experience and I loved working on Zulip. I got to learn a lot about python, mypy, several libraries(both standard and third-party) and many other things. I started contributing in March 2016 and from then to 20 August 2016, I have the second highest number of commits(commit graph, commit list) in Zulip!
I’m very grateful to my mentor, Tim Abbott. He has been very responsive and helpful and it was great working with him. Zulip has a vibrant community and I love being a part of it.
Some of our scripts require listing files of a particular type. For example, pyflakes requires listing all python files and our linters for plain text files, markdown, CSS, JavaScript, etc. require listing files of appropriate types. We usually have to include extensionless scripts in the list(based on shebang line) as well and we need a mechanism for excluding certain files which we don’t want listed for some reason. There didn’t used to be a centralized way to do this listing.
For mypy and check-py3(check-py3 is explained in the python 3 section), we needed to list all python files. So it was decided that we should write a module whose function is to list files of a particular type. I wrote that module and named it lister.py .
I wrote a script which runs mypy in python 2 mode on most of our python files. I added to the exclude list all the files which didn’t pass the mypy check. I also made tools/run-mypy run in Travis CI.
Even when we don’t annotate our code, mypy can still infer a lot about our code and sometimes find a few bugs. So our strategy was to first make as many files passthe mypy check(in python 2 mode) as possible and then start annotating them.
Annotate files
This constitutes a big part of my work. I learned a lot about python while doing this.
Many people added mypy type annotations during PyCon sprints and I was one of the reviewers of pull requests sent by them.
I mostly focused on the core of Zulip, that is zerver/lib . While annotating, I also fixed a few bugs I encountered. Sometimes mypy also revealed some bugs in Zulip. Most of these bugs were about fixing string types. This helped us a lot when we were adding python 3 support because most of the compatibility issues were because of incorrect string handling.
Run mypy in python 3 mode
#1164 - After we had done some work towards python 3 compatibility, we thought it would make sense to run mypy in python 3 mode.
#1172 - Unannotated files had to be edited a bit to make them pass mypy check in python 3 mode.
This mostly helped detect bugs where we were using standard library functions which were no longer supported in python 3. I also found out some bugs in typeshed itself and fixed those by sending pull requests to typeshed.
Zulip’s development environment now works on python 3, with the small exceptions of tools/run-dev.py(because it uses twisted). We also have all tests running on python 3(except the production deployment test suite).
Dependencies needed to be upgraded because older versions of dependencies were not python 3 compatible.
Other python 3 compatibility issues, like using standard library functions no longer supported by python 3.
Improved py3k
This document explains what is py3k. py3k was not run on all python files; I fixed that. I also made it faster. This made our code syntactically python 3 compatible.
Mypy type annotations
tools/lister.py
Add tools/run-mypy
Shrink mypy exclude list
Annotate files
Run mypy in python 3 mode
Run mypy on extensionless python scripts
Add Python 3 compatibility
Main challenges
Improved py3k
Make tools/lint-all runnable on python 3
Use a virtualenv in production