TravisCI: Pandas version advanced (Jan 2021) numpy is now incompatible

Question

I am very much a TravisCI beginner. That said, I maintain python package mplfinance and we run a TravisCI test suite for every Pull Request. We run the same tests for py36, py37 and py38. Beginning in January 2021, the Py37 tests began failing consistently with the error:

ERROR: pandas 1.2.0 has requirement numpy>=1.16.5, but you'll have numpy 1.16.4 which is incompatible.

Here's the thing: Some of the tests run using pytest under tox, and those tests are fine. But we also run a test from a shell script. In that test, for each PR, the script checks out and installs the package (using pip3 install .) first from the PR itself, and then from the version currently in the repository, and does some comparison of the two versions. This works fine for py36 and py38, and before January worked fine also for Py37. Here is an example of it working fine for Py37 prior to January:

https://travis-ci.org/github/matplotlib/mplfinance/jobs/751128518

In the above link you can see the following

Collecting pandas
  Using cached pandas-1.1.5-cp37-cp37m-manylinux1_x86_64.whl (9.5 MB)
Collecting python-dateutil>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting cycler>=0.10
  Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Requirement already satisfied: numpy>=1.15 in /home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages (from matplotlib->mplfinance==0.12.7a4) (1.16.4)

Notice that in installs pandas 1.1.5 and numpy 1.16.4

Beginning in January this is what I see:

Example: https://travis-ci.org/github/matplotlib/mplfinance/jobs/753110408

Collecting pandas
  Using cached pandas-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (9.9 MB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.3.1-cp37-cp37m-manylinux1_x86_64.whl (1.1 MB)
Collecting pillow>=6.2.0
  Using cached Pillow-8.1.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
Requirement already satisfied: numpy>=1.15 in /home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages (from matplotlib->mplfinance==0.12.7a5) (1.16.4)

Now it is installing pandas 1.2.0, and initially it appears satisfied with numpy 1.16.4, but then later in the file I see

ERROR: pandas 1.2.0 has requirement numpy>=1.16.5, but you'll have numpy 1.16.4 which is incompatible.

(Oddly, further up in the file, it appears that tox is installing numpy==1.19.4 and pandas==1.1.5 ??)

Does anyone have any ideas what I can do about this?

Much appreciated. Thank you.

Answer 1

This turned out to be a problem with pip determining the correct dependency: for some reason it thought that the pandas requirement was for numpy>=1.15

The error message

ERROR: pandas 1.2.0 has requirement numpy>=1.16.5 ...

was coming from pandas itself (upon import).

The fix is to upgrade to a newer version of pip (inside the script running on travisCI). The newer version of pip is able to corrently detect that pandas needs a newer version of numpy.

Just put pip install --upgrade pip near the top of the script.

Not sure why TravisCI doesn't just use the latest pip by default.

Reference: https://travis-ci.community/t/11214