Python Criticism from a Julia Perspective
A Julia developer reflects on the experience of using Python 3.10
Python feels eerily familiar to Julia developers. That is not an accident. Python is one of the languages which inspired the creation of Julia. From the creators of Julia:
We are power Matlab users. Some of us are Lisp hackers. Some are Pythonistas, others Rubyists, still others Perl hackers.
Both languages can be used as script languages with interactive REPL (read-evaluate-print-loop) environments, interfacing elegantly with many other programming languages and environments.
In fact, that ability to act as an elegant glue language has been one of the main selling points for Python. When I began working as a professional developer over 20 years ago, Python was one of the first languages I picked up for doing text processing and code generation. At the time, Python had no rival in terms of being able to write clean programs parsing and manipulating text.
Perl's programmers would have objected loudly at the time, but where is Perl today? The focus on clean and easy to read code won over the flexibility at almost any cost that Perl pursued. Perl ended up as a write-only language. It was great and writing code, but not for reading.
# Excerpt from The 3rd Annual Obfuscated Perl Contest
undef $/;open(_,$0);/ \dx([\dA-F]*)/while(<_>);@&=split(//,$1);@/=@&;
$".=chr(hex(join("",splice(@&,0,2))))while(@&); eval$”;
When I began using Python, it felt like a far more obscure thing than Julia is today. Look where Python is today. Python is no longer the odd kid in the corner but the 800 pound Gorilla dominating machine learning, data science and web development. Judging by Stackoverflow's 2022 Developer survey, the only general purpose programming language more popular than Python today is JavaScript.
But I must confess that Python in 2022 does not give me the same positive vibes that Python in the year 2000 did. Why? Python capability has certainly grown exponentially. But so has our expectations from a programming language.
Sophisticated IDEs such as PyCharm and big frameworks such as TensorFlow, Django and Pandas let Python developers do a lot of stuff. I will not reflect upon that, but rather the out-of-the-box experience of the language itself.
Complexity of Modern Python
Today's Python is not your grandfathers Python. You had the Python 2.x vs 3.x problem, Anaconda vs PIP, pipenv, virtualenv, venv, JAX vs PyPy vs Numba, Cython and the list goes on. Enough people have felt the pain for it to have made it into a comic strip.
I once had a consulting assignment with a medical doctor wanting to build an application to help with her diagnosis. She had some basic experience with programming and jumped into the most obvious route for beginners today: Python.
But is Python still the obvious gold standard for ease of use in programming? I don't think so. The good doctor came to me because she eventually had to give up dealing with the complexity of the Python package system and virtual environments. She stumbled across Julia and found it a breath of fresh air.
Managing Virtual Environments in Python and Julia
To deal with the Python 2.x vs 3.x complexity and installation of packages of different versions, we need a virtual environment. However, Python was not made for virtual environments. It is something that got bolted on. Thus, there are many ways of creating virtual environments. I happen to use one called VirtualFish which is made to work with the Fish Shell which I use in my Terminal app. There is a problem right there: Depending on the Shell environment you use, you require different software to manage your virtual environment.
What exactly is a virtual environment in Python? It involves creating an elaborate Python directory structure for each virtual environment, with links to concrete versions of different libraries and Python version. Here is an example from VirtualFish. I use the vf new command to create a new virtual environment called rockets. You can see that it creates countless files and folders under the .virtualenvs/ directory. We will create a structure like this for every virtual environment.
❯ vf new rockets
Creating rockets via /usr/local/opt/python@3.8/bin/python3.8 …
~ via 🐍 v3.8.12 (rockets)
❯ tree -L 3 .virtualenvs/
.virtualenvs/
└── rockets
├── bin
│ ├── activate
│ ├── activate.csh
│ ├── activate.fish
│ ├── activate.ps1
│ ├── activate.xsh
│ ├── activate_this.py
│ ├── easy_install
│ ├── easy_install3
│ ├── easy_install3.8
│ ├── pip
│ ├── pip3
│ ├── pip3.8
│ ├── python -> /usr/local/opt/python@3.8/bin/python3.8
│ ├── python3 -> python
│ ├── python3.8 -> python
│ ├── wheel
│ ├── wheel3
│ └── wheel3.8
├── lib
│ └── python3.8
└── pyvenv.cfg
There are hooks in the shell to make sure your whole system points to the directory structure for whatever virtual environment you have made current. Thus, installing a virtual environment manager implies adding stuff to your shell configuration. I have the following file added to my fish startup configuration:
~
❯ cat .config/fish/conf.d/virtualfish-loader.fish
set -g VIRTUALFISH_VERSION 2.5.1
set -g VIRTUALFISH_PYTHON_EXEC /usr/local/opt/python@3.8/bin/python3.8
source /usr/local/lib/python3.8/site-packages/virtualfish/virtual.fish
emit virtualfish_did_setup_plugins⏎
This way of creating virtual environments are not unproblematic. The system can crash with other tools. There are various virtual environment systems you have to know whether you can use together or not. Packaging and deploying this kind of stuff is not trivial.
Managing virtual environments in Julia in contrast is borderline boring. An environment is just a directory with two files:
Project.toml- Direct dependenciesManifest.toml- Indirect dependencies
That means you can easily version control an environment in Julia. You just have two files you need to bring to recreate that environment anywhere. Let me show you how we create an environment in Julia for comparison. I will create an environment called troll (why not? I am Norwegian. We name everything something with trolls).
❯ mkdir troll
❯ cd troll
~/troll
❯ julia -q
julia>
We have started up Julia, and we are in the Julia REPL (read-evaluate-print-loop). It is currently in Julia mode, indicated by the prompt being julia>. We switch to package manager mode to add dependencies to an environment. We just hit the ] key to switch mode.
julia> ]
(@v1.7) pkg> activate .
Julia gives us some feedback on how and where it is fetching dependencies.
We don't have to actually exit the Julia REPL. We can simply hit the semicolon ; to get into shell mode, and we can use regular Unix utilities like cat to look at the .toml files which just got modified. The Project.toml file contains the single dependency we added:
shell> cat Project.toml
[deps]
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
shell> cat Manifest.toml
# This file is machine-generated - editing it directly is not advised
julia_version = "1.7.2"
manifest_format = "2.0"
[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
We could, of course, go into details about what all these different things mean in the Manifest file, but that isn't what this story is about. What we are trying to do is to compare contemporary Python with Julia.
There are several important takeaways from the two systems we have just explored:
Python requires picking a particular virtual environment system compatible with the shell you use and install it.
Installing a virtual environment system in Python modifies the behavior of your shell.
For Julia, there is nothing to install. The virtual environment system comes as an integral part of Julia and interwoven with the packaging system. You activate and modify virtual environment systems from within the Julia REPL itself. It does not modify or touch your shell at all. You don't have to worry about adding a virtual environment system messing up other shell configurations you may have made.
You easily carry virtual environments with your projects in Julia, as they are just simple text files you can commit to your version control system and keep in the same directory as your project.
I don't see an easy way for Python to dig itself out of the mess it has created in this space. Past mistakes live on because there are a huge number of Python projects out there built on these systems. Adding a new a better system creates a well-known problem all software developers are familiar with:
Problems with Python Documentation
One of the most frequent complaints about Julia is that the documentation sucks, while Python is praised for its abundance of great documentation. I believe this is a mischaracterization.
Julia has a really well-thought-out documentation system. The Julia REPL offers a help mode, which you enter by pressing ?. In help mode, actual Julia code is evaluated to pick the relevant method. For instance, the multiplication symbol in Julia could be used for multiplying the numbers or concatenating strings. If multiply two numbers in help mode, we get the documentation for the arithmetic operation.
help?> 2 * 4
*(x, y...)
Multiplication operator. x*y*z*... calls this function with all
arguments, i.e. *(x, y, z, ...).
Examples
≡≡≡≡≡≡≡≡≡≡
julia> 2 * 7 * 8
112
julia> *(2, 7, 8)
112
If on the other hand, we supply to text strings we get documentation for string concatenation instead:
help?> "hello" * "world"
*(s::Union{AbstractString, AbstractChar}, t::Union{AbstractString, AbstractChar}...) -> AbstractString
Concatenate strings and/or characters, producing a String. This is
equivalent to calling the string function on the arguments.
Concatenation of built-in string types always produces a value of type
String but other string types may choose to return a string of a
different type as appropriate.
Examples
≡≡≡≡≡≡≡≡≡≡
julia> "Hello " * "world"
"Hello world"
julia> 'j' * "ulia"
"julia"
The Python REPL documentation is nothing of this level of sophistication. But the Python documentation has bigger problems:
No syntax highlight or formatting
No example code
Extremely sparse
To illustrate the problem, I took some screen shoots of what looking up help for similar types of functions/methods would produce in each REPL environment, Julia on the left and Python on the right.
Print String
Let us look at the print documentation first. That is the most commonly used function for a beginner to the language. Notice how the Julia help gives helpful code example and gives important details about how non-string values will get formatted. Python says nothing about how print will treat non-string values.
Push and Append Collection
Another common functionality for any beginner would be to add an element to the end of a collection. Julia calls this function push! while Python calls it append. Again, you can see how spartan the Python documentation is. It is just stating the most obvious without any details or code examples.
The help documentation also illustrates what, I think, is another clear advantage to a more functional language like Julia. The help documentation immediately expose you to the fact that push! Is a generic function that applies to numerous collection types. You are also informed about possible differences in how it behaves for different collections. The OOP approach of Python gives you a much more isolated world. You cannot immediately tell whether this method applies to many different collection types or not. Julia helps you to think in generic terms. You start thinking about functions as having related behavior across multiple types.
Split String
Both Python and Julia are good language for text manipulations. A function I use a lot is split to turn a string into an array of elements. Again, the Julia variant has far richer documentation with more examples to help the developer. For instance, the Julia documentation elaborates on what the delimiter dlm can be. The documentation explains that it can be a string, regular expression, function, single character or collection of characters.
The Python documentation doesn't actually say what kind of object is expected for the separator sep.
Why Does Julia Documentation Have a Bad Reputation, Then?
Many Julia projects are young and immature, which means the documentation is incomplete or lacking. Python in contrast have many large and mature projects which have built up solid documentation over time.
The discrepancy in the impressions stems from the fact that Python a wide variety of books, tutorials, guides, and online documentation available. The Julia community have much fewer resources and have simply not created proper documentation for all popular projects. But there are some important things Julia got right where Python comes up short: The built-in documentation system for Julia is clearly superior. It understands Markdown syntax, you can add code examples and all of this can be pulled out with documentation generation tools. That means the help you see in the Julia REPL and what you see online will match up.
For Python, these are two separate worlds. The help for the split method was very spartan in the Python REPL, but is far more extensive on the web based online help system.
If you look at the same documentation (split) for Julia, you will find it is identical to what you find in the Julia REPL. That is because the documentation is sourced from the same place.
Weakly Typed Booleans Expressions
A common operation in many languages is to check if a collection xs is empty. Many languages have functions for this purpose named things like empty, isempty or isEmpty. In Julia, I would write:
julia> xs = [3, 4];
julia> isempty(xs)
false
I remember when first getting back to Python after many years, I had forgotten how to check if a collection was empty, so I tried a variety of method calls with the mentioned names. None of them worked because, surprise, surprise, in Python to check if a collection xs is empty you write:
if not xs:
print "collection is empty"
This oddity exists because the Python world has the concept of truthy and falsy. In Python, pretty much every object which isn't 0, 0.0, false or an empty collection is interpreted as true. That is in my view a rather bad design choice. It gives very weak typing for boolean expressions and control-flow statements. Weak typing means more bugs slip through the cracks.
Perhaps more puzzling about this design choice is that it completely violates one of the most sacred commandments in the Zen of Python:
Explicit is better than implicit
This language design choice means that the following clearly buggy code runs fine:
if 'B':
print("expression is true")
In Julia, if you tried to run this expression, Julia would throw a type error exception:
julia> if 'B'
print("true expression")
end
ERROR: TypeError: non-boolean (Char) used in boolean context
Low Function Discoverability
If you have programmed for a number of years, you start developing a sense of what sort of functionality should exist. You know that it should be possible to add items to a collection, split up strings, search for an item. In Julia, right out of the box, if I write spli and hit tab twice I get the following completions shown:
splice! split splitdir splitdrive splitext splitpath
julia> spli
That is a lot of useful information. I am reminded that I can split a path into its individual components, or that I can split off the filename extension.
julia> splitext("foo.txt")
("foo", ".txt")
With Python, searching for functionality is more complex because it is divided across free functions and methods on objects.
Keep reading with a 7-day free trial
Subscribe to Erik Explores to keep reading this post and get 7 days of free access to the full post archives.