dbldatagen.datasets.basic_geometries module

class BasicGeometriesProvider[source]

Bases: NoAssociatedDatasetsMixin, DatasetProvider

Basic Geometry WKT Dataset

This is a basic process geospatial dataset with WKT strings representing POINT, LINESTRING, and POLYGON geometries.

It takes the following options when retrieving the table:

  • random: if True, generates random data

  • rows : number of rows to generate

  • partitions: number of partitions to use

  • geometryType: one of point, lineString, or polygon, default is polygon

  • maxVertices: number of points in each lineString or polygon

As the data specification is a DataGenerator object, you can add further columns to the data set and add constraints (when the feature is available)

Note that this datset does not use any features that would prevent it from being used as a source for a streaming dataframe, and so the flag supportsStreaming is set to True.

ALLOWED_OPTIONS = ['geometryType', 'maxVertices', 'random']
COLUMN_COUNT = 2
MAX_LOCATION_ID = 9223372036854775807
MIN_LOCATION_ID = 1000000
getTableGenerator(sparkSession, *, tableName=None, rows=-1, partitions=-1, **options)[source]

Gets data generation instance that will produce table for named table

Parameters:

  • sparkSession – Spark session to use

  • tableName – Name of table to provide

  • rows – Number of rows requested

  • partitions – Number of partitions requested

  • autoSizePartitions – Whether to automatically size the partitions from the number of rows

  • options – Options passed to generate the table

Returns:

DataGenerator instance to generate table if successful, throws error otherwise

Implementors of the individual data providers are responsible for sizing partitions for the datasets based on the number of rows and columns. The number of partitions can be computed based on the number of rows and columns using the autoComputePartitions method.