_2022-10-15

CVE-2007-4559: Directory traversal vulnerability

First, I want to make clear that I do not speak on behalf of the Python development team or in any other official capacity. I have stepped down as the tarfile maintainer in 2019, so this statement is purely personal.

In my opinion the claims that there is a security vulnerability in the tarfile module that has been ignored for 15 years are somewhat exaggerated and out of context.

However, I recognize that programmers may unwittingly run into problems when they use the tarfile module with archives from external sources, because they are not aware of the possible security implications. That is why there is a prominent warning in the official module documentation.

To be clear, it is completely safe to access tar archives from dubious origins with the tarfile module unless all of the following criteria are met:

There are no security measures in place to make sure the archive is safe to extract, i.e. inspection, verifying a cryptographic signature, sandboxing etc.
The archive is extracted to the filesystem in large parts or in its entirety without specific selection of members. (With tarfile it is possible to access members as file-like objects without them being extracted to the filesystem.)
The archive is extracted to the filesystem with unrestricted user privileges.

Context

Most importantly, the tarfile module is a library. It serves as a basic building block for programmers to use in their own projects in order to build applications for end-users. It is not a stand-alone tar program like GNU tar or bsdtar that have a different and much larger audience. This is why I think many requirements that without a doubt apply to a tar program do not necessarily apply to the tarfile module. tarfile is neither intended nor designed to be used directly. Between tarfile and the end-user there is always the programmer whose responsibility it is to make informed decisions with regard to functionality, application and security.
It is not safe to extract tar archives with tarfile that originate from untrusted and possibly malicious sources. This is clearly stated in the official module documentation. However, the harm that can be done is confined within the boundaries that are set by the operating system, i.e. the privileges of the current user. In any case, it is never a good idea to handle potentially malicious data in the context of superuser privileges.
The tarfile module does exactly what it is supposed to do when it is executing the instructions stored in the tar archive. The alleged security vulnerability in tarfile is not the result of a mistake or negligence. tarfile does what it does by design. I wrote some more on that here in 2014.
I believe it is necessary and unavoidable that programmers that use other people's code in their projects have to take responsibility to some degree. They should make themselves familiar with what the code does, how it works and what the possible risks are. They should take the trouble to read at least some amount of the documentation.
It is understandable that many programmers do not think about possible security implications when they use tarfile. But I think tarfile as a library should not impose too many restrictions on how it can be used and what it can be used for. In the end it is a tool, and tools can be dangerous if used improperly. Also, for a library it is hard to tell if the programmer is doing something on purpose or by accident. A library must be able to rely on the programmer knowing what he is doing. For example, it should not be a database's job to check for SQL injection attacks and prevent malicious queries from being executed. These kinds of things are up to the programmer.

The Trellix Story

This blog post from Kasimir Schulz of Trellix shows a good example but not for a security vulnerability in tarfile but for very flawed application design.

As far as I understand it, the blog post illustrates a vulnerability of the Spyder IDE which uses the tar format for sharing and transferring a state of variables between different projects and people. The tarfile module is used to read these tar archives.

However, this feature is implemented very poorly:

It is no good idea to feed data or code into your program from sources that are not trustworthy without prior inspection or other sufficient security measures.
In the example code, tarfile is used to extract the entire archive to the filesystem although only one file from the archive is actually needed. It would be much safer, more efficient and less error-prone to use tarfile's extractfile() method to read the file's data directly from the tar archive without extracting anything to the filesystem.
The extracted file contains a set of pickled Python objects which are read into memory. The documentation for the pickle module explicitly states that it is not secure to unpickle data from sources you do not trust. Even if you take tarfile out of the equation, Spyder IDE is still vulnerable to attacks via pickle.

Therefore, this blog post does not show a security vulnerability in the tarfile module but instead in the Spyder IDE. Both the tarfile and the pickle modules are used in ways they are not supposed to be used and that are strongly discouraged in the documentation.

It is peculiar that this example is used to attest the tarfile module a security vulnerability but not the pickle module.

Conclusion

I understand the community's desire to fix this issue no matter how. Unfortunately, fixing these kinds of issues is not trivial and requires a lot of effort.

After dismissing the first bug report in 2007, I proposed a patch for discussion in the Python bug tracker in 2014. At that time, it seemed to me that this was not the way most of the people wanted the problem to be fixed. The discussion instantly died down, so there was no clear vote and the patch was never implemented.

In 2018, the discussion about the patch was resumed, but due to time constraints I was no longer able to participate. I had increasing difficulty fulfilling my role as tarfile maintainer. Therefore, in 2019 I gave up my position as maintainer.

/home/lars/cve-2007-4559

CVE-2007-4559: Directory traversal vulnerability

Context

The Trellix Story

Conclusion

References