Wpull requires the following:
- Python 3.2 or greater
- Or lxml for faster but much worse HTML parsing
- Or cchardet for faster version of chardet
The following are optional:
- psutil for monitoring disk space
- Lunatic Python (bastibe version) for Lua support
- Manhole for a REPL debugging socket
- youtube-dl for downloading complex video streaming sites
For installing Wpull, it is recommended to use pip installer.
Wpull is officially supported in a Unix-like environment.
Once you have installed Python, lxml, and pip, install Wpull with dependencies automatically from PyPI:
pip3 install wpull
--upgrade option will upgrade Wpull to the latest
--no-dependencies to only upgrade Wpull.
--user option will install Wpull into your home
Automatic install is usually the best option. However, there may be outstanding fixes to bugs that are not yet released to PyPI. In this case, use the manual install.
Install the dependencies known to work with Wpull:
pip3 install -r https://raw2.github.com/chfoo/wpull/master/requirements.txt
Install Wpull from GitHub:
pip3 install git+https://github.com/chfoo/wpull.git#egg=wpull
as the path will install Wpull’s develop branch.
psutil is required for the disk and memory monitoring options but may not be available. To install:
pip3 install psutil
Wpull has pre-built binaries located at https://launchpad.net/wpull/+download. These are unsupported and may not be up to date.
Please obtain the latest Python release from http://python.org/download/ or your package manager. It is recommended to use Python 3.3 or greater. Versions 3.2, 3.3, 3.4 are officially supported.
PyPy 2.3.1 (Python 3.2 implementation) is supported.
Python 2 is not supported.
It is recommended that lxml is obtained through an installer
or pre-built package. Windows packages are provided on
https://pypi.python.org/pypi/lxml. Debian/Ubuntu users
python3-lxml. For more information, see
If pip is not installed on your system yet, please follow the instructions at http://www.pip-installer.org/en/latest/installing.html to install pip. Note for Linux users, ensure you are executing the appropriate Python version when installing pip.
To enable optional Lua scripting support, Lunatic Python (bastibe version) can be installed using pip:
pip3 install git+https://github.com/bastibe/lunatic-python.git#egg=lunatic-python
At time of writing, Lunatic Python uses Lua 5.2. If you desire a different version of Lua, please see below.
At time of writing, Lunatic Python does not support Python 3.2.
The version of Lunatic Python in the Python Package Index (PyPI) is out of date.
Specify Lua version¶
Download lunatic-python from https://github.com/bastibe/lunatic-python using
the “Download ZIP” link or
LUAVERSION to reflect the current Lua library
installed. On Debian/Ubuntu it is known by
Run pip to install Lunatic Python with
LOCATION replaced with the
location of the Lunatic Python source code.:
pip install LOCATION