Include GitHub repositories to requirements.txt
Posted: ,
Modified:
Copyright certificate by blockai
Summary
This document explains how to include packages hosted in GitHub
but not not registered in PyPI into requirements.txt,
and how to configure setup.py referring requirements.txt.
requirements.txt
Supposing https://github.com/rgmining/common as the GitHub repository to be added to requirements.txt,
the package information for the repository is
-e git+https://github.com/rgmining/common.git#egg=rgmining_common-0.9.0
The format after #egg= shoule be <package name>-<version>.
If you’re using pip-tools to compile requirements.txt,
your requirements.in should have the same clause.
setup.py
To include requirements.txt in setup.py,
give the following load_requires_from_file function to install_requires attribute:
from setuptools import setup, find_packages
def load_requires_from_file(filepath):
with open(filepath) as fp:
return [pkg_name.strip() for pkg_name in fp.readlines()]
setup(
# Other clauses are omitted.
install_requires=load_requires_from_file("requirements.txt")
)
If your requirements.txt has URLs,
load_requires_from_file function should be the following:
def take_package_name(name):
if name.startswith("-e"):
return name[name.find("=")+1:name.rfind("-")]
else:
return name.strip()
def load_requires_from_file(filepath):
with open(filepath) as fp:
return [take_package_name(pkg_name) for pkg_name in fp.readlines()]
and give those URLs via dependency_link attribute in setup function:
def load_links_from_file(filepath):
res = []
with open(filepath) as fp:
for pkg_name in fp.readlines():
if pkg_name.startswith("-e"):
res.append(pkg_name.split(" ")[1])
return res
setup(
# Other clauses are omitted.
install_requires=load_requires_from_file("requirements.txt"),
dependency_links=load_links_from_file("requirements.txt"),
)
Finally, python setup.py test downloads packages from GitHub and runs tests.