Dependencies for the Data Generator framework

Building the framework will install a number of packages in the virtual environment used for building the framework. Some of these are required at runtime, while a number of packages are used for building the documentation only.

The following packages are used in building the test data generator framework. All packages used are already installed in the Databricks runtime environment for version 6.5 or later

• numpy==1.22.0

• pandas==1.2.5

• pickleshare==0.7.5

• py4j==0.10.9

• pyarrow==4.0.1

• pyspark>=3.1.3

• python-dateutil==2.8.1

• six==1.15.0

• pyparsing==2.4.7

• jmespath==0.10.0

The following packages are required for development only

• wheel==0.36.2

• setuptools==52.0.0

• bumpversion

• pytest

• pytest-cov

• pytest-timeout

• rstcheck

• prospector

The following packages are only required for building documentation and are not required at runtime

• sphinx==5.0.0

• sphinx_rtd_theme

• nbsphinx

• numpydoc==0.8

• pypandoc

• ipython==7.22.0

• recommonmark

• sphinx-markdown-builder

• Jinja2 < 3.1

• sphinx-copybutton